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FOREWORD 


Troubleshooting  Assessment  and  Enhancement  (TAE)  Program  (previously  titled 
Troubleshooting  Proficiency  Evaluation  Program,  TPEP)  was  sponsored  by  the  Deputy  Chief  of 
Naval  Operations  (OP- 11)  and  was  performed  under  ()603720N-R1772-ET01.  The  purpose  of  the 
TAE  program  was  to  develop  a  low-cost  microcomputer- based  system  to  provide  an  objective 
measure  of  the  troubleshooting  proficiency  of  Navy  technicians. 

This  technical  note  is  the  last  of  three  that  document  the  TAE  program.  It  presents  findings  from 
the  test  and  evaluation  effort  conducted  at  the  Advanced  Electronics  School  Depanment,  Service 
Schools  Command,  Navy  Training  Center,  San  Diego,  California.  Using  troubleshooting  episodes 
developed  and  presented  on  the  7  AE  computerized  delivery  system,  students  and  instructors  at  the 
school  were  assessed  on  the’r  troubleshooting  performance.  Results  were  determined,  conclusions 
drawn,  and  recommendations  made.  Recommendations  focus  on  enhancing  the  effectiveness  of  the 
current  TAE  delivery  system  and  the  corresponding  troubleshooting  episodes. 

The  first  technical  note  presents  the  results  of  the  literature  survey,  the  theoretical  and 
methodological  issues  that  were  to  be  considered,  and  the  proposed  test  and  evaluation  plan  for  the 
TAE  effort  (Conner  &  Hassebrock,  1991).  The  second  technical  note  presents  the  design  and 
development  of  the  computerized  troubleshooting  proficiency  evaluation  system  (Conner,  Poirier, 
Ulrich,  &  Bridges,  1991). 


J.  C.  McLACULAN 
Director,  Training  Systems  Depanment 
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SUMMARY 


Problem 

The  Navy  has  limited  means  of  measuring  the  troubleshooting  proficiency  of  Navy  technicians 
and  their  ability  to  contribute  to  operational  readiness.  There  is  limited  capability  to  maintain  or 
enhance  troubleshooting  skills  on-board  ships  or  at  Reserve  Readiness  Centers,  or  to  evaluate  over¬ 
all  troubleshooting  capability  so  that  feedback  can  be  provided  to  the  training  command  to  improve 
troubleshi Hiring  skills  training  in  the  schools. 

Purpose 

The  purpose  of  the  Troubleshooting  Assessment  and  Evaluation  (TAE)  program  was  to  devel¬ 
op  a  low-cost,  microcomputer-based  system  to  provide  an  objective  measure  of  the  troubleshooting 
proficiency  of  Navy  technicians. 

The  TAE  effort  resulted  in  the  development  of  a  troubleshooting  proficiency  demonstration  for 
the  maintainers  (NEC  ET-1453s)  of  the  Naval  Modular  Automated  Communications  System(V)/ 
Satellite  Communications  (NAVMACS  V/SATCOM)  hardware.  The  test  and  evaluation  plan  was 
designed  to  assess  the  TAE  system,  validate  the  TAE  troubleshooting  episodes,  and  assess  the  re¬ 
liability  and  effectiveness  of  the  episodes  in  evaluating  peifoimance  of  troubleshooting  techni¬ 
cians. 

Method 

The  TAE  episodes  were  constructed,  the  troub'eshooting  evaluation  factors  were  determined 
and  weighted,  the  research  subject  groups  were  defined,  and  the  specific  research  objectives  were 
delineated.  Seven  research  areas  with  22  hypotheses  were  identified. 

The  TAE  test  was  administered  to  students  in  the  system  phase  of  the  course  ani  to  qualified 
instructors  (NEC  ET-1453)  in  a  classroom  at  the  school.  The  subjects  went  through  a  learn  pro¬ 
gram,  2  practice  episodes  and  14  test  troubleshooting  episodes. 

Demographic  and  performance  data  were  collected  for  53  students  and  ’  instructors.  The  data 
were  evaluated  for  completeness,  descriptive  statistics  were  inspected,  and  any  deficiencies  or 
anomalies  were  resolved.  The  data  were  analyzed  to  relate  TAE  performance  to  hypotheses  in  the 
seven  research  areas;  experience,  electronics  knoi^ledge,  electronics  performance  proficiency,  dif¬ 
ficulty  level  (of  troubleshooting  episodes),  time,  (use  of)  comple;,.  test  equipment,  and  ranking  (of 
subjects) 

Results  and  Conclusions 

1 .  TTiere  was  no  general  statistically  significant  relationship  between  experience  and  TAE  trou¬ 
bleshooting  fierformance  on  the  two  hypotheses  tested.  One  hypothesis  was  dropped  due  to 
lack  of  fleet  subjects. 

2.  There  was,  generally,  no  consistent  relationship  between  electronics  knowledge  and  TAE  per¬ 
formance  on  the  three  hypotheses  tested. 
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3.  There  was  a  general  and  tonsistent,  significant  relationship  between  the  electronics  perfor¬ 
mance  proficiency  measures  and  TAE  performance  on  the  six  hypotheses  tested. 

4.  There  was  a  general  and  consistent,  significant  relationship  between  the  difficulty  of  the  epi¬ 
sodes  and  the  TAE  performance  on  the  three  hypotheses  tested.  One  hypothesis  was  dropped 
due  to  the  lack  of  fleet  subjects. 

5.  There  was  a  general  and  consistent,  significant  relationship  between  lime  and  TAE  results  on 
the  two  hypotheses  tested. 

6.  There  was  no  significant  relationship  between  test  equipment  usage  and  TAE  performance  re¬ 
sults  on  the  one  hypothesis  tested. 

7.  There  was  no  significant  relationship  between  the  subject  rankings  and  TAE  performance  on 
the  three  hypotheses  tested. 

Future  Efforts 

The  following  future  efforts  are  recommended: 

1.  Further  investigate  TAE  as  related  to  validity  and  reliability. 

2.  Further  investigate  TAE  as  related  to  experience  and  troubleshooting  performance. 

3.  Further  investigate  TAE  as  related  to  “academic”  and  “knowledge”  factors. 

4.  Further  investigate  the  relationship  between  selection  requirements  and  troubleshooting  perfor¬ 
mance. 

5.  Further  analyze  the  TAE  data  and  results  to  improve  the  discriminatory  and  predictive  accuracy 
of  the  TAE  approach. 

6.  Further  test  the  TAE  approach  on  other  subjects  and  on  other  equipment  and  equipment  types. 

7.  Develop  additional  troubleshooting  episodes  to  provide  directive  training,  guided  training,  and 
tests  with  feedback. 
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INTRODUCTION 


Problem 

Currently  the  Navy  has  limited  means  of  objectively  measuring  the  troubleshooting 
proficiency  of  shipboard  technicians  and  their  ability  to  contribute  to  operational  readiness.  Other 
than  subjective  supervisory  opinion,  there  is  no  consistent  and  reliable  way  to  assess  the  transfer 
of  training,  particularly  hands-on  training  on  hardware  systems  provided  in  Navy  “C”  schools. 
Once  the  “C”  school  graduate  has  been  integrated  into  the  ship’s  force,  fleet  commanders  have  no 
comprehensive  method  to  assess  the  technician’s  performance  capabilities  or  skill  degradation 
over  time  In  addition,  the  schools  receive  no  quantifiable  feedback  identifying  specific  areas 
where  troubleshooting  training  requires  greater  emphasis  or  improvement. 

Due  to  limited  availability  of  system  hardware  at  “C”  schools,  actual  hands-on  training  time 
is  severely  restricted.  This  limits  the  amount  of  time  students  explicitly  use  their  system  knowledge 
and,  therefore,  decreases  the  effectiveness  of  instructional  programs.  Once  on-board,  the  ship 
safety  hazards  associated  with  corrective  maintenance  of  weapon  system  hardware  preclude  the 
use  of  drill  and  practice  exercises.  This  limits  the  technician’s  ability  to  maintain  or  improve 
troubleshooting  skills. 

Purpose 

The  purpose  of  the  Troubleshooting  Assessment  and  Evaluation  (TAE)  program  was  to 
develop  a  low-cost,  microcomputer-based  system  to  provide  an  objective  measure  of  the 
troubleshooting  proficiency  of  Navy  technicians. 

The  TAE  effort  resulted  in  the  development  of  a  troubleshooting  proficiency  demonstration 
for  the  high-technology  (electronic/digital)  maintainer  community  (NEC  ET-1453)  for  the  Naval 
Modular  Automated  Communications  System(V)/Satellite  Communications  (NAVMACS(V)/ 
SATCOM)  hardware  system. 

This  repiort  documents  the  test  and  evaluation  of  the  TAE  demonstration  in  the  training 
environment.  Specifically,  the  test  and  evaluation  were  designed  to  (1)  assess  the  TAE 
troubleshooting  evaluation  and  diagnostic  factors,  (2)  validate  the  ability  of  the  TAE  episodes  to 
evaluate  and  diagnose  troubleshooting  proficiency,  and  (3)  assess  the  reliability  and  effectiveness 
of  the  TAE  episodes  to  evaluate  troubleshooting  proficiency,  diagnose  results,  and  lead  to 
improved  training  and  performance. 

Background 

The  specific  objectives  of  the  TAE  project  were  to  support  the  Navy  operational  and  training 
communities  by  providing  a  microcomputer-based  system  to  (1)  assess  personnel  troubleshooting 
capabilities  within  the  Navy  training  environment  (e.g.,  “C”  school  and/or  reserve  training 
activities),  (2)  develop  drill  and  practice  for  personnel  in  training  awaiting  hardware  availability  or 
active  duty  assignments,  (3)  improve  curricula  and  training  methods  based  on  school 
troubleshooting  assessment  results,  (4)  provide  fleet  and  reserve  on-board  training  (OBT)  through 
drill  and  practice  exercises,  (5)  develop  an  objective  measure  of  operational  readiness  of  fleet  and 
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reserve  personnel  in  their  area  of  systems  hardware  troubleshooting  capability,  (7)  improve 
operational  readiness,  and  (8)  improve  curricula  and  instructional  methods  as  a  result  of 
operational  fleet  and  reserve  feedback  of  assessment/evaluation  data  to  the  training  community. 

The  ultimate  benefits  to  be  realized  as  a  result  of  the  TAE  program  include  fl)  improvement 
of  operational  readiness  defined  in  terms  of  reduction  in  Mean  Time  To  Repair  (MTTR),  increase 
in  Mean  Time  Between  Failures  fMTBF),  reduction  in  No  Fault  Removals  (NFRs)  (i.e.,  return  of 
unfailed  pditsl,  (2)  improvement  of  curricula  and  instructional  methods  as  a  result  of  consistent  and 
objective  operational  fleet  feedback  of  assessment/evaluation  data  to  the  training  community,  and 
(3)  improvement  of  the  training  community’s  ability  to  objectively  evaluate  student 
trouble shv^Hjting  performance. 

The  TAE  effort  was  organized  into  three  phases;  design,  development,  and  test  and 
evaluation.  The  phases  included  the  following  activities: 

1.  Selection  of  the  NAVMACS(V)/SATCOM  hardware  system  and  the  NEC  ET-145? 
maintainer  community  for  the  demonstration. 

2.  Review  of  the  literature  to  provide  input  into  the  design  and  development  of  the 
troubleshooting  episodes  and  the  test  and  evaluation  procedures. 

3.  Design  and  development  of  computer  software  to  support  the  evaluation  program. 

4.  Design  and  development  of  the  troubleshooting  episodes  selected  as  representative  for  the 
demonstration  maintenance  community. 

5.  Design  and  development  of  training  assessment  and  training  drill  and  practice  episodes. 

6.  Design  and  development  of  a  troubleshooting  episode  development  capability  to  be  used  for 
other  hardware  systems. 

7.  Development  of  factors  for  evaluating  troubleshooting  proficiency. 

8.  Development  of  a  test  and  evaluation  plan  stating  the  research  hypotheses  and  analysis 
techniques. 

9.  Data  collection,  analysis,  and  reporting  for  the  test  and  evaluation. 

The  review  of  literature  and  a  discussion  of  the  theoretical  and  methodological  issues  in  TAE 
design  and  the  initial  test  and  evaluation  plan  (Steps  1  and  2)  are  documented  in  Conner  and 
Hasiebrock,  1991);  the  TAE  computer  software  design  and  development  efforts  (Steps  3  through 
6),  in  Conner,  Poirier,  Ullrich,  and  Bridges  (1991). 

The  following  paragraphs  (1)  define  the  TAE  demonstration  problem  set,  (2)  describe  the 
development  of  the  evaluation  factors  for  troubleshooting  proficiency,  (3)  define  the  test  and 
evaluation  subject  groups,  (4)  identify  the  research  objective  of  the  test  and  evaluation,  and  (5) 
state  the  TAE  research  hypotheses.  The  remaining  sections  of  the  report  describe  the  method, 
results,  conclusions  and  recommendations  of  the  TAE  demonstration  test  and  evaluation. 
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Appendixes  A  and  B  provide  detailed  test  and  evaluation  data.  Appendixes  C  and  D  provide  copies 
of  the  questionnaires  used  in  the  development  and  weighting  of  the  TAE  factors. 

TAE  Episodes 

Within  the  context  of  the  TAE  demonstration,  troubleshooting  is  viewed  as  pan  of  the 
corrective  maintenance  function.  When  a  system  is  not  functioning  properly,  corrective 
maintenance  must  be  performed  to  return  the  system  to  an  optimum  operational  state. 
Troubleshooting  is  the  means  by  which  the  faulty  component  of  the  system  is  identified.  Once 
identified,  the  faulty  components  can  be  repaired/replaced.  Figure  1  displays  this  lelationship.  The 
TAE  epi.sodcs  were  designed  to  measure  the  ability  to  troubleshoot  by  identifying  the  faulty 
component 
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Figure  1.  Hardware  activity  to  troubleshooting. 

The  TAE  testing  format  begins  by  displaying  fault  indicators.  The  subject  uses  a  series  of 
menus  to  review  fault  symptoms,  front  panels,  maintenance  panels,  and  diagnostic  information;  to 
select  equipment,  to  make  reference  designator  tests  or  replace  a  Lowest  Replaceable  Unit  (LRU). 
The  subject’s  goal  in  the  TAE  test  is  to  find  the  faulty  LRU  as  defined  by  the  mauitenance 
philosophy  of  the  system.  This  is  done  by  selecting  the  suspected  LRU  for  replacement.  It  is 
possible  for  the  fault  symptom  to  logically  lead  to  an  LRU  that  is  not  the  faulty  LRU  as  defined  by 
the  episode.  This  is  indicated  as  a  GOOD  FAULT  but  not  the  specific  faulty  LRU. 

TTie  troubleshooting  assessment  episodes  were  developed  for  seven  of  the  eigh* 
NAVMACS(V)/SATCOM  subsystems  are  listed  below.  No  TAE  episodes  were  developed  for 
troubleshooting  the  TSEC/KG-36  due  to  the  sensitivity  and  classification  problems  associated  with 
this  subsystem. 
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AN/USH-26(V)  Subsystem 

1 .  Formatter  A 

2.  Formatter  B 

3.  Servo/Data 

4.  Parallel  Interface 

5.  Control 

AN/L’SO  Subsystem 

1.  Maintenance  Panel  Keyboard 

2.  Power  Supply 

3.  CRT 

4.  2nd.  3rd  Page  RAM 

5.  Microcontroller 

AN/UYK-20(V)  Subsystem 

1.  Channel  16  Interface 

2.  Micro  Channel  15  and  lO  Oneshot  Control 

3.  Channel  14  Interface 

4.  Memory  Interface 

5.  Memory  Interface 

CV-3333AJ  Subsystem 

1 .  Sample  Processor  Assembly 

2.  Sample  Data  Generator  Assembly 

3.  Spectrum  Analyzer  No.  2 

4.  Handset 

5.  Analyzer  and  Synthesizer  Analog 

6.  Voicing  and  Channel  Encoder 

7.  Pitch  Analyzer 

8.  Spectrum  Analyzer  No.  2 

9.  Timing  and  Interface 

10.  Timing  and  Self  Test 

ON-143(V)AJSQ  Subsystem 

1 .  Level  Converter 

2.  Transmit  Sequence  Control 

3.  Relay  Card 

4.  Rec  Synchronization 

5.  Red'Black  Interface 

6.  Red/Black  Interface  Relay 

RD-397U  Subsystem 

1.  Punch  Enable  Signal 

2.  LDR  Signal 

3.  OD  3  Signal 
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TT-624(V)5AJG  Subsystem 

1.  Input  &  Buffer  Data  Registers 

2.  Hammer  Drivers 

3.  Paper  Feed  Control  Lx)gic 

4.  Output  Decode 

5.  Serial  Interface  Logic 

The  7AE  testing  episodes  developed  for  the  demonstration  may  be  used  as  troubleshooting 
training  exercises  as  well  as  troubleshooting  assessment  tools.  There  are  five  demonstration 
systems  at  the  Fleet  Training  Center,  Norfolk  and  six  demonstration  systems  at  the  Advanced 
Etectronic  Schools  Department,  Service  Schools  Command,  San  Diego  for  training  and  evaluation 
purposes  Although  the  test  and  evaluation  plan  focuses  on  the  ability  of  TAE  to  assess 
troubleshooting  proficiency,  TAE  should  also  be  viewed  in  the  broader  context  as  an  instructional 
tool. 


In  addition  to  these  testing  episodes,  three  other  levels  of  TAE  episode  presentation  were 
planned:  directive  training,  guided  training,  and  test  with  feedback.  However,  only  14  directive 
training  episodes  were  developed  (listed  below)  and  no  guided  training  episodes  or  tests  with 
feedback  were  produced. 

ANAJSH-26(V)  Subsystem 

1.  Servo  Data 

2.  Control 

AN/USQ-69(V)  Subsystem 

1.  2nd,  3rd  Page  RAM 

2.  Microcontroller 

AN/UYK-20(V)  Subsystem 

1.  Card  Location  J06 

2.  Card  Location  A24 

CV-3333/U  Subsystem 

1.  Spectrum  Analyze 

2.  Synchronization,  Control  Logic 

ON-143(V)/USQ  Subsystem 

1 .  Rcc  Synchronization 

2.  Transmit  Sequence 

RD-397L  Subsystem 

1 .  Punch  Driver  Assy 

2.  Reader  Controller 

TT-624(V)5/UG  Subsystem 

1 .  Output  Decode 

2,  Serial  Interface  Logic 
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The  directive  training  episodes  are  designed  so  that  the  student  is,  in  effect,  looking  over  the 
shoulder  of  an  expert  troubleshooter  as  a  fault  is  discovered.  The  symptoms  are  provided  and  then 
information  is  presented  on  ( 1 )  what  the  symptoms  should  tell  the  troubleshooter,  (2)  what  tests  or 
checks  should  be  made,  and  (3)  what  conclusions  could  be  drawn  from  these  tests  or  checks.  Then, 
a  test  or  check  is  accomplished.  The  results  of  the  test  or  check  are  displayed,  and  the  implication 
of  that  check  or  test  are  provided.  This  sequence  is  continued  until  the  fault  is  identified. 
Throughout  the  sequence,  the  student  observes  the  activity  and  follows  the  action  in  the  technical 
manual'  H ’'^s).  Information  and  graphics  from  the  TMs  are  provided  in  the  presentation  as 
appropnate 

Troubleshooting  Evaluation  Factors 

As  stated  earlier,  the  focus  of  the  TAE  episodes  is  fault  diagnosis.  The  Navy  electronics 
training  schools  identified  the  following  six  steps  in  the  fault  diagnosis  process  (Conner,  1987); 

1.  Symptom  Recognition.  The  technician  determines  if  there  is  a  fault  by  checking  system 
outputs. 

2.  Symptom  Elaboration.  The  technician  checks  all  possible  indicators,  built-in  test  features, 
refining  the  likely  list  of  symptoms. 

3.  Probable  Faulty  Functions.  The  technician  narrows  the  list  of  faults  for  testing  purposes. 

4.  Localizing  the  Faulty  Function(s).  The  technician  checks  outputs  at  test  points  to  further 
eliminate  areas  or  functions. 

5.  Isolating  the  Faulty  Circuit.  The  technician  narrows  the  fault  suspect(s)  to  a  particular 
circuit  component. 

6.  Failure  Analysis.  The  technician  determines  why  malfunction  occurred. 

The  microcomputer-based  TAE  episodes  do  not  specifically  test  the  first  step,  symptom 
recognition,  or  the  last  step,  failure  analysis.  A  fault  indication  is  represented  in  the  scenario  and 
the  student  solves  the  problem  to  the  lowest  replaceable  unit  Actual  replacement  of  the  unit  in 
order  to  repair  the  failure  involves  motor  skills  that  are  not  easily  simulated.  Nor  is  there  any 
attempt  to  query  students  as  to  the  reason  the  fault  would  have  given  the  symptoms. 

Steps  2  through  5  in  the  fault  diagnosis  process  required  a  further  breakdown  to  determine 
measurable  troubleshooting  proficiency  factors.  As  part  of  the  Troubleshooting  Proficiency 
Evaluation  Program  for  the  NATO  Seasparrow  Surface  Missile  System,  a  panel  of  25  experts 
(three  civilian  engineers  and  22  Navy  personnel  in  five  different  technical  ratings)  representing  the 
Mobile  Training  Unit  (MOTU),  the  “C”  school,  and  the  Shore  Intermediate  Maintenance  Activity 
(SIMA)  were  assembled  to  further  define  the  performance  factors  (Conner,  1986, 1987).  Based  on 
previous  research  and  the  expertise  of  the  group,  the  following  nine  factors  were  identified: 

1.  Solution 

2.  Time 

3.  Cost 
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4.  Proof  Points 

5.  Out  of  Bounds 

6.  Test  Points 

7.  Valid  Check 

8.  Invalid  Check 

9.  Illogical  Approach 

For  li.e  TAE  demonstration,  a  questionnaire  describing  the  nine  performance  factors  as  they 
relate  to  the  evaluation  of  troubleshooting  skills  was  developed.  The  questionnaire  was 
disseminated  to  1200  Navy  high-technology  maintenance  personnel  in  technical  environments  and 
in  the  fleet.  Respondents  were  asked  to  complete  a  background  information  form  and  then  rank 
order  the  lactors  in  order  of  importance.  Since  the  relative  importance  of  the  factors  may  change 
with  conditions,  the  following  conditions  were  assumed:  (1)  non-combat,  (2)  normal  day  in  home 
port,  and  (3)  trouble  was  encountered  during  a  normal  systems  check.  A  total  of  750  questionnaires 
were  returned  and  analyzed  to  rank  the  nine  factors.  A  copy  of  the  questionnaire  and  the  analysis 
is  contained  in  Appendix  C. 

Based  on  the  results  of  the  fust  quesrionnaire,  a  second  questionnaire  was  administered  to 
subject  matter  expens  in  troubleshooting  Navy  systems  and  equipment.  The  respondents  were 
provided  a  list  of  the  nine  factors  in  rank  order  and  asked  to  determine  the  weight  that  should  be 
applied  to  each  factor.  A  total  of  45  quesdonnaires  were  returned  and  analyzed  to  assign  weights. 
A  copy  of  the  questionnaire  and  analysis  is  contained  in  Appendix  D. 

Once  the  factors  were  ranked  and  weighted  as  shown  in  Table  1,  the  next  step  was  to  develop 
a  “rational”  scheme  for  scoring  the  episodes.  Given  that  the  intent  of  the  scoring  is  to  discriminate 
between  levels  of  troubleshooting  proficiency,  it  was  determined  that  failure  to  solve  the  problem 
would  result  in  a  score  of  0,  solving  the  problem  would  result  in  a  score  of  100.  There  would  be  no 
partial  score  for  factor  1.  Therefore,  the  ability  to  discriminate  between  levels  of  troubleshooting 
proficiency  would  be  based  on  the  scoring  the  remaining  factors. 

Table  1 


Ranking,  Weighting,  and  Scoring  Scale  for  Troubleshooting  Evaluation  Factors 


Rank 

Factor 

Weight 

Scoring  Scale 

1 

Solution 

42.78 

100.00 

2 

Cost  (Incorrect  Solution) 

13.13 

23.98 

3 

Time 

11.80 

20.62 

4 

Proof  Points 

9.86 

17.23 

5 

Illogical  Approach 

6.87 

12.01 

6 

Invalid  Checks 

4.68 

8.18 

7 

Out  of  Bounds 

4.00 

6.99 

8 

Number  of  Tests 

3.21 

5.61 

9 

Number  of  Checks 

3.08 

5.38 
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Since  the  activities  that  a  subject  performs  during  the  episode  are  being  recorded  for  further 
analysis  of  behavioral  protocols,  the  data  are  available  to  develop  a  scoring  scale  to  use  for  factors 
2  through  9.  The  weights  for  the  factors  were  converted  into  a  scale  equaling  100  points  as  shown 
in  Table  1.  The  final  score  equals  100  points  minus  the  sum  of  the  points  for  each  factor.  The 
minimum  score  is  0  points.  There  are  no  negative  scores.  The  scoring  criteria  for  each  factor  is 
listed  below: 


1.  Solution 

2.  Cosi 

3.  Time 

4.  Proof  Points 

5.  Illogical  Approach 

6.  Invalid  Checks 

7.  Out  of  Bounds 

8.  Number  of  Tests 

y.  Number  of  Checks 


-100  for  failure  to  discover  fault 

-5  for  each  no  fault  replacement  to  maximum  of  -23.98 

-.5  X  total  minutes  to  maximum  of  -20.62 

-points  based  on  number  of  proof  points  to  maximum  of  -17.23 

-6  for  each  illogical  approach  to  maximum  of  -12.01 

-.8  X  number  of  invalid  checks  to  maximum  of  -8.18 

-.6  X  number  of  out  of  bounds  to  maximum  of  -6.99 

-.5  X  number  of  tests  to  max  of  -5.61 

-.5  X  number  of  checks  to  max  of  -5.38 


The  factors  listed  below  were  incorporated  into  the  TAE  episodes  as  measures  to  evaluate 
and  diagnose  an  individual’s  troubleshooting  proficiency  level.  The  cost  factor  in  the 
questionnaires  was  changed  to  incorrect  solutions  to  more  accurately  describe  the  actual  behavior. 
As  a  result  of  the  previous  literature  search  (Conner  &  Hassebrock,  1991),  another  factor  to  record 
redundant  checks  was  also  added  to  the  TAE  scoring  criteria. 


1.  Correct  Solution  indicates  the  troubleshooting  problem  is  correctly  solved,  i.e.,  the  faulty 
component  is  identified. 

2.  Incorrect  Solutions  indicate  the  number  of  LRUs  identified  as  the  faulty  component  that 
were  not  faulty. 

3.  Total  Time  is  the  total  minutes  from  login  to  logout  that  it  takes  the  subject  to  find  the  fault. 

4.  Test  Points  are  the  total  valid  reference  designator  tests. 

5.  Proof  Points  are  test  points  that  positively  identify  LRUs  as  faulty.  Generally  there  will  be 
at  least  two  proof  points  associated  with  an  LRU,  an  input  and  an  output  point. 

6.  Invalid  Checks  indicate  an  inappropriate  test  was  performed  at  an  appropriate  test  point.  For 
example,  a  subject  measures  current  where  he  should  have  been  checking  for  voltage. 

7.  Valid  Checks  indicate  an  appropriate  piece  of  test  equipment  was  used  at  a  test  point.  For 
example,  a  subject  measures  current  where  current  should  be  measured. 

8.  Redundant  Checks  indicate  the  same  test  was  made  at  the  same  test  point  at  some  time 
during  the  episode. 
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9.  Out-of-Bounds  indicates  an  inappropnate  test  point  was  selected  An  example  vsoiiki  be  tl.c 
selection  of  a  test  point  that  is  not  reasonably  in  the  area  of  where  the  tremble  is  located 

10.  Illogical  Approaches  indicate  an  inappropnate  equipment  selection  occurred.  For  example, 
the  'abject  begins  testing  on  L'NIT  7,  xxhen  all  the  symptoms  and  indications  are  that  the  fault 
IS  with  UNIT  1  and  the  solution  could  be  accomplished  starting  at  UNIT  1. 

1  provides  a  pictorial  representation  of  the  TAE  approach  as  well  as  the  factor 

relatlo^lshlp,^  One  of  the  problems  in  the  past  in  developing  “real  hardware”  troubleshooting 
episcnie^  o  the  number  and  type  of  tests  and  test  points  that  must  be  provided  if  the  "entire” 
universe  of  troubleshooting  is  available.  If  all  options  are  made  available  for  even  one  fault 
symptom  die  amount  of  computer  memory  required  is  extremely  large.  Given  the  intent  of  TAE 
is  to  as.scss  troubleshooting  proficiency,  only  the  test  points  dictated  by  the  sy  mptom  were  included 
in  the  "approved"  list  of  test  points.  This  was  done  to  keep  the  required  number  of  test  points  to  a 
manageable  set  (i.e..  a  “troubleshooting  spectrum”)  with  all  others  to  be  recorded  as  “out  of 
bounds.”  The  test  point  set  for  a  troubleshooting  epi^ode  was  further  reduced  by  the  factor  of 
illogical  approach.  The  rationale  here,  given  the  intent  of  TAE,  was  that  even  though  one  could 
ultimately  arrive  at  a  problem  solution  and  the  point  of  departure  was  "w'ithin  bounds.”  the 
approach  was  not  appropriate  given  the  symptom.  Further,  given  the  input-conversion-output 
concept  of  electronic  circuitry,  some  tests  that  MUST  be  miade  to  "prove”  a  component  is  faulty 
(i.e.,  proof  points).  Also,  given  a  set  of  fault  symptoms,  it  is  postulated  that  there  is  an  "optimum 
path”  (i.e.,  senes  of  tests  that  should  be  made)  to  the  faulty  component.  The  current  fonn  of  TAE 
has  developed  the  capability  to  evaluate  all  factors  as  described.  However,  it  has  not  developed  the 
"optimum  path”  as  yet.  This  was  expected  to  be  accomplished  empirically  once  data  were 
collected. 


System  Troubleshooting 

Troubleshooting  Universe 


Troubleshooting  Episode 


Set  of 
Fault 

■ 

Troubleshooting 

Symptoms 

I 

Spectrum 

*  Fault 


Illogical  Approach 


O  Optimum  Path  □  Out  of  Bounds 
#  Proof  Points  □  In  Bounds 


F  igure  2.  TAE  Factors  Model. 
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Research  Subject  Groups 

The  test  and  evaluation  plan  specified  three  different  subject  groups.  The  groups  were 
identified  as  novice,  experienced,  and  expert.  Within  the  novice  group,  two  sets  of  individuals 
should  show  similar  performance  scores:  (1)  “C”  school  students  (at  end  of  course)  rnd  (2) 
apprentice/inexperienced  individuals  that  have  graduated  from  “C”  school  and  held  a  NEC  ET- 
1453  rating  for  less  than  one  year.  Individuals  with  less  than  one  year  experience  are  considered 
novices  sin,,c  fleet  personnel  do  not  have  sufficient  opportunity  to  work  on  their  specific  system  to 
become  “experienced”  until  they  have  been  aboard  ship  for  a  year  or  more.  Journeymen/ 
experienced  personnel  are  defined  as  individuals  who  hold  a  NEC  ET-1453,  are  currently  assigned 
to  a  ship  with  NAVMACS  system.  ET-1453  billets,  and  have  been  working  for  more  than  one  year 
(on  their  specific  system).  Masters/experts  are  defined  as  individuals  who  hold  a  NEC  ET-1453 
and  have  one  year  or  more  of  experience  working  at  a  special  technical  assignment  such  as  a 
Mobile  Technical  Unit  (MOTU)  or  as  a  technical  representative  at  a  comparable  project  office.  An 
expert  could  be  a  school  instructor,  a  NAVMACS  MOTU  representative,  or  project  office 
engineering/technical  support  military  staff  member  for  NAVMACS(V)/SATCOM  system 

Research  Objectives 

The  purpose  of  the  test  and  evaluation  was  to  provide  information  on  the  reliability  and 
validity  of  the  TAE  demonstration  to  discriminate  between  levels  of  troubleshooting  proficiency. 
To  establish  the  objectives  of  the  test  and  evaluation,  it  is  necessary  to  define  the  terms  reliability 
and  validity  within  the  TAE  context. 

Reliability  concerns  the  problem  of  errors  in  measurement.  For  TAE,  the  focus  is  on  whether 
an  individual’s  scores  are  consistent  across  TAE  episodes  and  whether  the  scores  on  TAE  episodes 
are  consistent  across  individuals. 

The  general  concept  of  validity  revolves  around  the  question,  “Does  an  instrument  perform 
the  function  it  is  intended  to  perform?”  For  TAE,  the  question  is  “Does  TAE  discriminate  between 
different  levels  of  troubleshooting  proficiency?”  Validity  is  usually  a  matter  of  degree  as  opposed 
to  an  all  or  nothing  propeny.  There  are  three  types  of  validity  to  investigate  within  the  context  of 
the  TAE  test  and  evaluation;  (1)  empirical  validity,  (2)  content  validity,  and  (3)  construct  validity. 

Empirical  validity  may  be  defined  in  terms  of  predictive  and  concurrent  validity.  Predictive 
validity  is  concerned  with  the  ability  of  a  test  to  predict  performance  in  the  future,  while  concurrent 
validity  is  concerned  with  the  relationship  between  the  test  and  a  contemporary  measure  of 
performance  (Conner,  1987).  For  TAE,  concurrent  validity  may  be  addressed  by  evaluating  TAE 
performance  against  “C”  school  jjerformance,  supervisor  ratings,  and  similar  measures. 

Predictive  validity  refers  to  the  ability  of  an  instrument  to  estimate  some  behavior,  known  as 
the  cntcnon.  For  TAE,  the  criterion  of  interest  is  troubleshooting  proficiency.  However,  an 
“ultimate”  criterion  measure  of  troubleshooting  proficiency  may  not  exist.  In  a  review  of  reports 
investigating  the  prediction  of  job  performance  of  military  enlisted  personnel,  Vineberg  and  Joyner 
(1982)  concluded  that  performance  in  training  is  currently  the  best  predictor  of  job  proficiency 
(measured  by  job  knowledge  tests)  and  job  performance  (measured  by  supervisor  ratings). 


10 


The  test  and  evaluation  plan  proposed  use  of  fleet  subject  matter  expens  and  instructor 
ratings  of  TAE  scoring  profiles  to  construct  a  troubleshooting  proficiency  criterion.  This  measure 
can  be  refined  over  time  to  produce  a  closer  approximation  of  the  ultimate  criterion  of 
troubleshooting  proficiency.  If  the  concepts  of  content  and  construct  validity  are  established,  it 
should  be  possible  to  build  a  strong  logical  connection  between  TAE  and  its  ability  to  predict 
troubleshooting  proficiency  of  electronics  technicians  in  the  fleet. 

Content  validity  is  concerned  with  the  question,  “Does  tlie  instrument  adequately  sample  a 
particular  domain?”  Content  validity  addresses  the  representativeness  of  the  content  of  the  test  and 
the  manner  in  which  it  is  presented  as  opposed  to  empirical  validity  (Nunnally  &  Durham,  1975). 
Issues  of  content  validity  were  incorporated  into  the  selection  of  problems  for  TAE  as  well  as  the 
design  of  the  TAE  episodes.  A  representative  sample  of  faults  was  identified  based  on  expen 
opinion  and  maintenance  information  available  on  the  NAVMACS(V)/SATCOM  system. 

Construct  validity  focuses  on  whether  the  content  domain  has  been  adequately  sampled  and 
developed  into  testing  measures  (Nunnally  &  Durham,  1975).  The  three  essentials  for  creating  and 
validating  a  construct  measure  are  to:  (1)  outline  the  domain  of  observables,  (2)  find  out  which  and 
how  much  the  different  observables  relate  with  each  other  or  are  affected  similarly  by  exjjerimental 
treatments,  and  (3)  find  out  whether  one  or  more  of  the  observable  variables  measures  the  construct 
(Nunnally  &  Durham,  1975). 

Troubleshooting  proficiency  is  viewed  as  a  multifaceted  attribute  realized  through  a  variety 
of  perceptual,  cognitive,  and  motor  behaviors.  The  domain  of  observable  measures  for 
troubleshooting  proficiency  encompasses  descriptive  measures  such  as  product  and  process; 
prescriptive  measures  such  as  cognitive  ability,  aptitude,  and  cognitive  style;  and  p)erformance 
dimensions  such  as  time,  errors,  and  inefficient  actions  (Conner,  1987).  Currently,  the  ten 
troubleshooting  evaluation  factors  represent  the  domain  of  observable  measures  for  TAE. 

Empirical  tests  can  determine  which  troubleshooting  measures  correlate  with  each  other  or 
are  similarly  affected  alike  by  TAE  treatments.  The  measures  that  respond  similarly  and 
consistently  for  the  different  treatments  hold  the  most  construct  validity.  The  TAE  test  and 
evaluation  plan  was  designed  to  investigate  whether  the  troubleshooting  factors  respond  in  a 
similar  and  consistent  manner  in  the  different  TAE  episodes.  Another  test  of  construct  validity  is 
whether  the  measures  of  the  construct  behave  as  expected.  Many  of  the  research  hypotheses  in  this 
effort  relate  to  the  construct  validity  of  TAE.  Taken  together,  they  serve  as  the  validation  of  TAE 
as  a  measurement  of  troubleshooting  proficiency. 

METHOD 


Research  Hypotheses 

The  research  hypnatheses  for  the  TAE  test  and  evaluation  are  organized  into  seven  categories: 
( I )  experience,  (2)  electronics  knowledge,  (3)  electronics  performance  proficiency,  (4)  difificulty 
level,  (5)  time,  (6)  complex  test  equipment,  and  (7)  ranking.  The  hypotheses  in  each  category  are 
described  below. 
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Experience  (Hypotheses  1  through  3) 

Previous  research  has  noted  that  experience  is  positively  correlated  with  troubleshooting 
proficiency  (Morris  &  Rouse,  1985).  The  TAE  subjects  are  organized  into  three  experience  levels 
(novice,  experienced,  expert).  The  following  hypotheses  were  designed  to  test  whether  the  TAE 
episodes  discriminated  between  troubleshooting  proficiency  by  the  experience  levels  of  subjects. 

1.  Instructors  (experts/masters)  will  score  significantly  higher  on  the  TAE  test  than  students 
(novices/apprentices). 

2  Experienced  fleet  personnel  journeymen  will  score  significantly  higher  on  the  TAE  test  than 
students  (novices). 

3.  Subjects  with  a  longer  time  in  the  electronics  rate  (i.e.,  time-in-service  [TIS])  experience  will 
score  significantly  higher  on  the  T.AE  test  than  subjects  with  less  time  in  that  rate 
(inexperienced). 

Hypothesis  1  was  tested  using  a  one-way  analysis  of  variance  (ANOVA)  that  compared  the 
group  means  of  the  TAE  test  scores  for  instructors  vs.  students.  No  data  were  available  for 
experienced  fleet  personnel  so  hypothesis  2  was  dropped  from  the  analysis.  Hypothesis  3  was 
tested  using  correlational  analyses.  The  following  data  were  required  to  test  each  hypothesis: 

1 .  AE  student  scores  and  TAE  instructor  scores. 

3.  TAE  student  scores,  TAE  instructor  scores,  and  time  in  service  (TIS)  for  all  subjects. 
Electronics  Knowledge  (Hypotheses  4  through  6) 

The  following  hypotheses  were  designed  to  test  the  concurrent  validity  as  well  as  the 
construct  validity  of  the  TAE  episodes  (in  this  case,  the  relationship  between  electronics 
knowledge  and  TAE  performance). 

4.  Students  (novices)  with  higher  academic  “C”  school  final  scores  will  score  higher  on  the 
TAE  test  than  students  (novices)  with  lower  scores. 

5.  Students  (novices)  with  higher  academic  “C”  school  subsystem  test  scores  will  score  higher 
on  the  TAE  subsystem  tests  (episodes)  than  students  (novices)  with  lower  “C”  school 
subsystem  test  scores. 

6.  Subjects  with  higher  appropriate  Armed  Services  Vocational  Battery  (ASVAB)  scores;  that 
is.  Electronic  Information  (El)  and  Electronics  Technician  selection  scores  (General  Science 
|GS]  Electronics  Information  [El]  -i-  Mathematics  Knowledge  [MK];  plus  Arithmetic 
Reasoning  [AR]  and  Armed  Forces  Qualification  Test  [AFQT])  will  score  higher  on  the  TAE 
test  than  subjects  with  lower  ASVAB  and  selection  scores. 

Hypotheses  4  through  6  were  tested  using  correlational  analyses.  The  following  data  were 
required  to  test  each  hypothesis: 
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4.  TAE  student  scores,  “C”  school  final  scores,  and  “C”  school  comprehensive  scores. 

5.  TAE  student  subsystem  scores  and  “C”  school  subsystem  scores. 

6.  TAE  student  scores  and  ASVAB  AFQT,  GS,  El,  MK,  and  AR  scores. 

Electronics  Performance  Proficiency  (Hypotheses  7  through  12) 

Previous  research  has  indicated  that  technical  knowledge  or  practical  job  knowledge  is 
related  to  troubleshooting  performance  (Morris  &  Rouse,  1985).  The  following  hypotheses  were 
designed  to  test  the  construct  validity  of  the  TAE  episodes  in  discriminating  between 
troubleshooting  proficiency  levels  (in  this  case,  the  relationship  between  specific  electronics 
troubleshooting  behavior  and  TAE  pierformance). 

7.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  invalid  checks 
than  less  proficient  subjects. 

8.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  illogical 
approaches  than  less  proficient  subjects. 

9.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  incorrect 
solutions  than  less  proficient  subjects. 

10.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  redundant  checks 
than  less  proficient  subjects. 

11.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  test  significantly  more  proof 
points  than  less  proficient  subjects. 

12.  In  general,  subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  significantly 
fewer  tests  than  less  proficient  subjects. 

Hypotheses  7  through  12  were  tested  using  correlational  analyses.  The  following  data  were 
required  to  test  each  hypotheses- 

7.  TAE  student  scores,  TAE  instructor  scores,  and  the  number  of  invalid  checks. 

8.  TAE  student  scores,  TAE  instructor  scores,  and  the  number  of  illogical  approaches. 

9.  TAE  student  scores,  TAE  instructor  scores,  and  the  number  of  incorrect  solutions. 

10.  TAE  student  scores,  TAE  instructor  scores,  and  the  number  of  redundant  checks. 

11.  T  AE  student  scores,  TAE  instructor  scores,  and  the  number  of  proof  points. 

1 2.  TAE  student  scores,  TAE  inso-uctor  scores,  and  the  number  of  the  following  types  of  tests: 
test  points,  equipment  selection  events,  front  panel  events,  maintenance  panel  events, 
fallback  test  events,  review  symptoms  events,  diagnostic  test  events,  load  operational 
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program  events,  step  procedure  events,  revision  events,  total  number  of  steps  taken  in  the 
episode. 

Difficulty  Level  (Hypotheses  13  through  16) 

It  seems  reasonable  to  assume  that  increasing  troubleshooting  task  difficulty  will  increase  the 
time  expended  (as  well  as  other  factors,  even  though  only  time  will  be  investigated  here)  in  finding 
the  solunon  The  length  of  time  to  solution  may  also  be  affected  by  the  subject’s  troubleshooting 
proficient)  level.  The  following  hypotheses  were  designed  to  test  the  reliability  of  the  TAE 
episodes  as  ss  ell  as  their  ability  to  discriminate  between  troubleshooting  proficiency  levels  (in  this 
case,  the  relationship  between  the  difficulty  of  troubleshooting  episodes  and  TAE  perform-ance. 

13.  The  more  difficult  the  episode,  the  longer  the  average  time  needed  to  find  the  solution  across 
subjects. 

14.  On  episodes  of  equal  difficulty,  subjects  with  a  higher  level  of  troubleshooting  proficiency 
will  take  significantly  less  time  than  less  proficient  subjects  in  finding  the  solution. 

15.  The  more  difficult  the  episode,  the  less  time  the  instructors  (experts)  will  take  to  find  the  TAE 
test  solutions  when  compared  to  the  students  (novices). 

16.  The  more  difficult  the  episode,  the  less  time  the  experienced  fleet  personnel  will  take  to  find 
the  TAE  test  solutions  when  compared  to  the  students  (novices). 

Hypotheses  13  through  15  were  tested  using  correlational  analyses.  No  data  were  available 
for  experienced  fleet  personnel  so  hypothesis  16  was  dropped  from  the  analysis.  The  following 
data  were  required  to  test  each  hypothesis; 

13.  TAE  difficulty  level  z  scores  for  each  episode  and  length  of  time  to  find  solution  for  each 
subject.  To  find  z  scores,  30  selected  TAE  variables  were  summed  across  all  subjects  for  each 
episode.  The  14  separate  episode  totals  were  then  transformed  into  z  scores  using  the 
Microstat  Statistical  Package.' 

14.  TAE  student  scores,  TAE  instructor  scores,  TAE  difficulty  level  z  scores  for  each  episode 
and  length  of  time  to  find  solution  for  each  subject. 

15.  TAE  student  scores,  TAE  instructor  scores,  length  of  time  to  find  solution  for  each  subject, 
difference  between  average  student  time  to  find  solution  and  average  instructor  time  to  find 
solution,  and  TAE  difficulty  level  z  scores  for  each  episode. 

Time  (Hypotheses  17  and  18) 

The  following  hypotheses  were  designed  to  validate  the  ability  of  the  TAE  episodes  to 
evaluate  and  diagnose  troubleshooting  proficiency  (in  this  case,  to  test  the  relationship  between 
time  and  the  TAE  episode  results).  Subject  matter  experts  noted  that  good  troubleshooters  often 
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14 


will  lake  a  longer  period  of  time  to  make  the  first  test  of  equipment.  This  observation  seems  related 
to  previous  research  concerning  cognitive  styles  and  troubleshooting  where  it  was  noted  that 
subjects  with  a  reflective  vs.  an  impulsive  cognitive  style  made  fewer  errors  in  troubleshooting 
tasks  (Morris  &  Rouse,  1985).  It  may  be  that  a  good  troubleshooter  begins  by  surveying  the  state 
of  the  equipment  to  generate  hypotheses  about  the  possible  fault,  uses  the  test  to  collect 
information,  and  then  takes  a  longer  amount  of  time  to  integrate  the  information  discovered  to 
generate  solutions  to  the  problem. 

17.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  take  significantly  less  total 
time  to  find  TAE  episode  solutions  than  less  proficient  subjects. 

18  In  general,  subjects  with  a  higher  level  of  troubleshooting  proficiency  will  take  a  significantly 
longer  time  than  less  proficient  subjects  before  making  the  first  test  point. 

Hypotheses  17  and  18  were  tested  using  correlational  aialyses.  The  following  data  were 
required  to  test  the  hypothesis: 

17.  TAE  student  scores,  TAE  instructor  scores,  and  total  time. 

18.  TAE  student  scores,  TAE  instructor  scores,  time  to  first  reference  designator  test,  and  time 
to  first  diagnostic  test. 

Complex  Test  Equipment  (Hypothesis  19) 

The  following  hypothesis  was  designed  to  test  the  construct  validity  of  the  TAE  episodes  (in 
this  case,  to  test  the  relationship  between  the  use  of  complex  test  equipment  and  TAE 
performance).  Previous  research  has  noted  that  good  troubleshooters  tend  to  make  more  difficult 
checks  than  do  poor  troubleshooters  (Morris  &  Rouse,  1985).  It  would  seem  reasonable  to  state 
that  good  troubleshooters  will  use  more  complex  test  equipment. 

19.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  significantly  more  tests 
using  an  oscilloscope  than  less  proficient  subjects. 

Hypothesis  19  was  tested  using  correlational  analyses.  The  following  data  were  required: 
TAE  student  scores,  TAE  instructor  scores,  and  the  number  of  oscilloscope  (waveform)  tests. 

Ranking  (Hypotheses  20  through  22) 

The  following  hypotheses  were  designed  to  test  both  the  concurrent  and  the  construct  validity 
of  the  TAE  episodes  (in  this  case,  the  relationship  between  student  rankings  and  performance 
indicators)  It  seems  reasonable  to  assume  that,  if  the  TAE  test  reflects  an  individual’s 
troubleshooting  proficiency,  there  will  be  a  positive  relationship  between  the  student’s  TAE  class 
ranking,  instructor  rankings,  and  “C’  school  course  ranking. 

20.  The  higher  the  student’s  TAE  class  rank,  the  higher  the  student  will  be  ranked  in  terms  of 
troubleshooting  proficiency  by  instructors/work  center  supervisors. 


15 


2 1 .  The  higher  the  student’s  TAE  class  rank,  the  higher  the  student’s  ranking  in  the  class. 

22.  The  higher  the  instructor  ranking  of  the  student  in  terms  of  troubleshooting  proficiency,  the 
higher  the  student’s  ranking  in  the  class. 

Hypotheses  20  through  22  were  tested  using  correlational  analyses.  The  following  data  were 
required  to  test  each  hypothesis: 

20.  Student  TAE  class  ranking  and  instructor  ranking  of  student  troubleshooting  proficiency. 

2 1  Student  TAE  class  ranking  and  course  class  student  ranking. 

22  Instructor  ranking  of  student  troubleshooting  proficiency  and  class  course  student  ranking. 

Test  Administration  Procedure 

Test  administration  was  conducted  by  Navy  Personnel  Research  and  Development  Center 
(NPRDC)  personnel  in  a  quiet  classroom  environment  at  the  Advanced  Electronics  Schools 
Department  (AESD),  Service  Schools  Command,  San  Diego,  California.  The  TAE  test  was 
administered  on  the  Zenith  248  microcomputer.  The  test  was  completed  using  technical 
documentation  for  the  NAVMACS(V)/SATCOM  (NEC  ET-1453)  system.  All  technical 
documentation  was  within  the  reach  of  the  subject  during  testing. 

The  test  administrator  assigned  the  subjects  to  one  of  two  randomized  test  sequences  to 
protect  the  TAE  study  from  test  order  effects.  A  total  of  16  episodes,  including  two  practice 
episodes,  were  administered  to  each  subject: 


1. 

AN/USH-26 

Formatter  A 

2. 

AN/USH-26 

Formatter  B 

3. 

ANAJSQ-69 

Power  Supply 

4. 

AN/USQ-69 

CRT 

5. 

ANA^YK-20 

Micro  Channel  15  and  lO  Oneshot  Control 

6. 

AN/VYK-20 

Channel  14  Interface 

7. 

CV-3333 

San-iple  Processor  Assembly 

8. 

CV-3333 

Sample  Data  Generator  Assembly 

9. 

ON- 143 

Transmit  Sequence  Control 

10. 

ON- 143 

Relay  Card 

11 

RD-397 

Punch  Enable  Signal 

12 

RD-397 

LDR  Signal 

13 

TT-62i 

Input  and  Buffer  Data  Registers 

14. 

TT-624 

Paper  Feed  Control  Logic 

15. 

TT-624 

Hammer  Drivers  (Practice) 

16. 

ANAJSH-26 

Parallel  Interface  (Practice) 

Each  episode  required  approximately  one  hour  (or  less)  to  complete,  although  there  was  no 
specific  time  limit.  The  subjects  completed  all  16  episodes  in  two  to  three  days.  The  test 
administrator  was  in  the  classroom  continuously  to  brief  subjects  and  set  up  the  programs. 
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Testing  began  with  a  brief  introduction  to  the  TAE  study  and  the  technical  documentation 
available.  The  subjects  read  and  signed  a  Privacy  Act  release  statement.  The  test  administrator 
staned  subjects  off  by  entering  their  social  security  number.  The  subjects  began  with  a  Learn 
Program,  followed  by  two  practice  episodes  to  become  familiar  with  the  TAE  test  displays  and 
menus.  After  testing  was  completed,  the  subjects  were  given  test  performance  feedl  ack. 

Subjects 

The  1 AE  test  and  evaluation  plan  was  designed  to  assess  the  troubleshooting  proficiency  of 
three  personnel  groups  within  the  Navy  electronics  training  and  shipboard  ensironments:  (1)  “C” 
school  students,  (2)  fleet  personnel,  and  (3)  personnel  designated  as  having  special  assignments. 
The  students  were  individuals  enrolled  in  “C”  school  during  the  “system”  phase  of  the  course.  The 
fleet  personnel  were  individuals  who  had  graduated  from  “C”  school,  held  tm  N’EC  ET-1453,  and 
had  varving  amounts  of  experience.  The  special  assignment  personnel  were  “C”  school  instructors 
who  taught  and  managed  training  in  the  electronics  classes  or  were  personnel  assigned  to  a  Mobile 
Technical  Unit  (\fOTU)  or  other  technical  assignments  dealing  with  the  NAVMACS(V) 
SATCOM. 

All  TAE  subjects  were  required  to  have  “C”  school  training  on  the  NAVMACS(V)/ 
SATCOM  subsystems.  For  students  enrolled  in  “C”  school,  the  tests  were  administered  in  the  last 
two  weeks  of  training  during  the  system  phase  of  the  course.  The  projected  number  of  subjects  for 
the  TAE  test  and  evaluation  was  approximately  100  students  (50  each  from  San  Diego  and 
Norfolk),  50  instructors  and  50  fieet  personnel  (25  each  from  each  coast). 

Data  Collection 

In  the  final  results,  data  were  collected  for  53  students  and  13  instructors,  all  at  AESD,  San 
Diego.  According  to  the  original  design,  student  and  instructor  data  were  also  going  to  be  collected 
at  Fleet  Training  Center,  Norfolk  and  from  fleet  personnel  from  both  coasts.  However,  due  to 
funding  cuts,  it  was  not  possible  to  gather  data  on  students  or  instructors  in  Norfolk  or  fleet 
personnel  on  either  coast.  As  a  consequence.  Hypotheses  7  and  16  were  dropped  from  the  study 
since  no  fleet  personnel  data  were  available.  In  other  hypotheses  dealing  with  time  or  experience, 
the  instructor  data  were  used  to  evaluate  the  hypotheses. 

Data  for  two  separate  data  bases  were  collected  by  NPRDC.  The  first  data  base  contained 
demographic  data  and  the  second  contained  TAE  program  performance  data.  Both  demographic 
and  TAE  performance  data  were  collected  for  seven  classes  of  “C”  school  students  between  April 
and  September  1989.  All  student  data  were  organized  by  “C”  school  class  number.  The 
dcnx)graphic  data  for  each  student  included:  Social  Security  Number,  time  in  service,  ASVAB 
scores,  “C  school  subsystem  final  scores,  “C”  school  comprehensive  score,  “C”  school  final 
score,  class  ranking,  TAE  ranking,  and  instructor  ranking.  Demographic  and  TAE  performance 
data  for  the  instructors  were  collected  during  September  1989.  The  demographic  data  for  each 
instructor  included  Social  Security  Number,  rate/rating,  time  in  service,  time  in  paygrade,  length 
of  time  holding  NEC  ET-1453,  length  of  time  working  on  the  NAVMACS(V)/SATCOM  system 
in  the  fleet,  and  length  of  time  as  a  NAVMACS(V)/SATCOM  instructor,  The  TAE  program  data 
for  both  students  and  instructors  consisted  of  scores  for  16  episodes  encompassing  6)3  variables. 
Thus,  each  subject,  whether  student  or  instructor,  received  673  separate  scores.  Table  2  presents 
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the  variables  for  each  case.  Table  3  describes  the  variables  for  each  episode  (Episode  1  is 
presented). 

As  the  demographic  data  and  TAE  performance  data  were  collected  by  NPRDC,  two  separate 
data  bases  were  built  using  the  Microstat  (R)  statistical  package.  The  TAE  performance  data  base 
included  all  of  the  data  collected  for  students  and  instructors.  The  data  in  the  data  base  were  refined 
before  files  were  created  for  performing  preliminary  evaluation  of  data.  First,  the  data  for  the 
practice  prunlems  were  dropped,  since  they  would  not  be  used  for  any  of  the  analyses.  Next, 
descriptive  .statistics  (mean,  standard  deviation,  minimum,  maximum)  were  calculated.  Then, 
based  on  the  descriptive  statistics,  data  were  dropped  for  variables  that  had  scores  of  zero  or 
standard  deviations  of  zero  across  all  students  and  instructors. 

Based  on  this  refined  data,  a  preliminary  evaluation  of  data  was  performed.  The  evaluation 
included:  (1)  inspecting  univariate  descriptive  statistics  for  accuracy  of  input,  (2)  evaluating 
number  and  distribution  of  missing  data,  (3)  identifying  and  dealing  with  outliers,  (3)  identifying 
and  dealing  with  skewness,  (4)  identifying  and  dealing  with  nonlinearity  and  heteroscedasticity, 
and  (5)  evaluating  variables  for  multicollinearity.  Based  on  this  preliminary  evaluation,  the  data  of 
five  students  were  dropped  due  to  missing  data.  The  data  of  two  instructors  were  also  dropped 
because  they  did  not  hold  NEC  ET- 1453.  Thus,  the  data  of  59  subjects  were  used  for  this  study,  48 
students  and  1 1  instructors. 

After  preliminary  data  evaluation  and  deletion  of  the  aforementioned  student  and  instructor 
data,  the  resultant  data  base  was  used  to  create  files  for  testing  the  study  hypotheses.  From  this 
refined  data  base,  a  matrix  file  was  created  with  all  the  variables  required  to  test  all  of  the  study 
hypotheses.  The  matrix  (Figure  3)  provides  an  overview  of  the  data  requirements  for  the  study 
hypotheses.  The  matrix  file  was  used  to  create  other  files  with  only  those  variables  specifically 
required  to  test  each  hypothesis. 


Table  2 


Description  of  TAE  Variables  for  Each  Case 


1 

3 

4 

5 

6 

7 

8 

Episode 

9  10 

11 

12 

13 

14 

LSH 

L'S069 

UYK20 

CV3333 

OS  143 

RD397 

TT624 

PRACTICE 

V2 

44 

86 

128 

170 

212 

254 

296 

338 

380 

422 

464 

506 

548 

590 

632 

Equipment 

(htrdware 

subsystem 

V? 

45 

87 

129 

171 

213 

255 

297 

339 

381 

423 

465 

507 

549 

591 

633 

Episode 

number 

V4 

46 

88 

130 

172 

214 

256 

298 

340 

382 

424 

466 

508 

550 

592 

634 

Found  Solu- 
tiar 

V5 

47 

89 

131 

173 

215 

257 

299 

341 

383 

425 

467 

509 

551 

593 

635 

Teft  Poinu 

V6 

48 

90 

132 

174 

216 

258 

300 

342 

384 

426 

468 

510 

552 

594 

636 

Oui-of- 

Bounds 

V7 

49 

91 

133 

175 

217 

259 

301 

343 

385 

427 

469 

511 

553 

595 

637 

Valid 

Checks 

V8 

50 

92 

134 

176 

218 

260 

302 

344 

386 

428 

470 

512 

554 

596 

638 

Invalid 

Checks 

V9 

51 

93 

135 

177 

219 

261 

303 

345 

387 

429 

471 

513 

555 

597 

639 

Redundant 

Checks 

VIO 

52 

94 

136 

178 

220 

262 

304 

46 

388 

430 

472 

514 

556 

598 

640 

Proof  Points 

VI 1 

53 

95 

137 

179 

221 

263 

305 

347 

389 

431 

473 

515 

557 

599 

641 

Toul  PPs 
Episode 

V12 

54 

96 

138 

180 

222 

264 

306 

348 

390 

432 

474 

516 

558 

600 

642 

Percenuge 

PPs 

V13 

55 

97 

139 

131 

223 

265 

307 

349 

391 

433 

475 

517 

559 

601 

643 

Toul  Time 

V14 

56 

98 

140 

182 

224 

266 

308 

350 

392 

434 

476 

518 

560 

602 

644 

TBD 

V15 

57 

99 

141 

183 

225 

267 

309 

351 

393 

435 

477 

519 

561 

603 

645 

Equip  Se¬ 
lect  Events 

V16 

58 

100 

142 

184 

226 

268 

310 

352 

394 

36 

478 

520 

562 

604 

646 

Front  Panel 
Events 

V17 

59 

101 

143 

185 

227 

269 

311 

353 

395 

437 

479 

521 

563 

605 

647 

Maint  Panel 
Events 

Vlg 

60 

102 

144 

186 

228 

270 

312 

354 

396 

438 

480 

522 

564 

606 

648 

Fallback 

Events 

V19 

61 

103 

145 

187 

229 

271 

313 

355 

397 

439 

481 

523 

565 

607 

649 

Ref  Desig 
Tests 

V20 

62 

104 

146 

188 

230 

272 

314 

356 

398 

440 

482 

524 

566 

608 

650 

Replace 

LRU  Events 

V21 

63 

105 

147 

189 

231 

273 

315 

357 

399 

441 

483 

525 

567 

609 

651 

Review 

Symp 

Events 

V22 

64 

106 

148 

190 

232 

274 

316 

358 

400 

442 

484 

526 

568 

610 

652 

TBD 

V23 

65 

107 

149 

191 

233 

275 

317 

359 

401 

443 

485 

527 

569 

611 

653 

Oiag  Test 
Evotts 

V24 

66 

108 

150 

192 

234 

276 

318 

360 

402 

444 

486 

528 

570 

612 

654 

Load  Op 
Prgm 

Events 

V2' 

f- 

>09 

151 

193 

235 

277 

319 

361 

403 

445 

487 

529 

571 

613 

655 

Step  Proced 
Events 

V2* 

68 

no 

152 

194 

236 

278 

320 

362 

404 

446 

488 

530 

572 

614 

656 

Revision 

Events 

69 

in 

153 

195 

237 

279 

321 

363 

405 

447 

489 

531 

573 

615 

657 

INC  Rep 
LRU  Events 

TBD  =  To  he  Jttcrmir.c4 
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Table  2  (Continued) 


I  2 

USH26 

3  4 

USQ69 

5  6 

UYK20 

7  8 

CV3333 

Episode 

9  10 

ON143 

11  12 
RD397 

13 

7X624 

14 

1 

PRACTICE 

V2« 

70 

112 

154 

196 

38 

280 

322 

364 

406 

448 

490 

532 

574 

616 

658 

Good  Ftuli 
Rep  LRU 
Events 

V29 

7: 

113 

155 

197 

239 

281 

323 

365 

407 

449 

491 

533 

575 

617 

659 

Time  to  1st 
Ref  Design 
Tests 

V}( 

114 

156 

198 

240 

282 

324 

366 

408 

450 

492 

534 

576 

618 

660 

Time  to  I  si 
Dug  Test 

V3l 

7^ 

1 15 

157 

199 

241 

283 

325 

367 

409 

451 

493 

535 

577 

619 

661 

Total  Steps 

V32 

74 

116 

158 

200 

242 

284 

326 

368 

410 

452 

494 

536 

578 

620 

662 

Wiveform 

Tesu 

V33 

75 

117 

159 

201 

243 

285 

427 

369 

411 

453 

495 

537 

579 

621 

663 

Voluge 

Tests 

V34 

76 

118 

160 

202 

244 

286 

328 

370 

412 

454 

496 

538 

580 

622 

664 

Read  Meter 
Tests 

V35 

77 

119 

161 

203 

245 

287 

329 

371 

413 

455 

497 

539 

581 

623 

665 

Logic  Tests 

V36 

78 

120 

162 

204 

246 

288 

330 

372 

414 

456 

498 

540 

582 

624 

666 

Current 

Tests 

V37 

79 

121 

163 

205 

247 

289 

331 

373 

415 

457 

499 

541 

583 

625 

667 

Frequency 

Tests 

V38 

80 

122 

164 

206 

248 

290 

332 

374 

416 

438 

500 

542 

584 

626 

668 

Contin¬ 
uity  Tests 

V39 

81 

123 

165 

207 

249 

291 

333 

375 

417 

459 

501 

543 

585 

627 

669 

Adjustment 

Tests 

V40 

82 

124 

166 

208 

250 

292 

334 

376 

418 

460 

502 

544 

586 

628 

670 

Final  Score 

V41 

83 

125 

167 

209 

251 

293 

335 

3r/ 

419 

461 

503 

545 

587 

629 

671 

TBD 

V42 

84 

126 

168 

210 

252 

294 

336 

378 

420 

462 

.504 

546 

588 

630 

672 

TBD 

V43 

85 

127 

169 

211 

253 

295 

337 

379 

421 

463 

505 

547 

589 

631 

673 

TBD 

TBD  =  To  be  dciermuied. 
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Table  3 


Variables  for  a  TAE  Episode 


Variable  Name 

Contents  of  Variable 

VI 

Subject’s  Social  Security  Number 

V2 

Equipment  (hardware  subsystem)  number  (1  =  USH26) 

V3 

Episode  number  (1) 

V4 

Found  Solution  (1  =  Yes,  0  =  No) 

v.s 

Number  of  Test  Points 

V6 

Number  of  Out  of  Bounds  tests 

V7 

Number  of  Valid  Checks 

V8 

Number  of  Invalid  Checks 

V9 

Number  of  Redundant  Checks 

VIO 

Number  of  Proof  Points  subject  tested 

Vll 

Total  number  of  Proof  Points  in  the  episode 

V12 

Percentage  of  proof  points  tested: 

(VIO  %  VI 1)  •  100,  rounded  to  a  whole  number 

V13 

Total  Time  spent  on  the  episode  (in  minutes) 

V14 

TBD 

V15 

Number  of  Equipment  Selection  events 

V16 

Number  of  Front  Panel  events 

V17 

Number  of  Maintenance  Panel  events 

V18 

Number  of  Fallback  test  events 

V19 

Number  of  Reference  Designator  test  events 

V20 

Number  of  Replace  LRU  events 

V21 

Number  of  Review  Symptoms  events 

V22 

TBD 

V23 

Number  of  Diagnostic  Test  events 

V24 

Number  of  Load  Operational  Program  events 

V25 

Number  of  Step  Procedure  events 

V26 

Number  of  Revision  events 

V27 

Number  of  INCORRECT  Replace  LRU  events 

V28 

Number  of  GOOD  FAULT  Replace  LRU  events 

V29 

Time  to  first  Reference  Designator  Test  (in  minutes) 

V30 

Time  to  first  Diagnostic  Test  (in  minutes) 

V31 

Total  number  of  steps  taken  in  the  episode:  ALL  events,  (even  “login”  and  “logout”)  except 
“revision”  events,  which  are  created  when  episode  data  is  edited  by  an  instructor. 

V32 

Number  of  Waveform  tests  performed 

V33 

Number  of  Voltage  tests  performed 

V34 

Number  of  Read  Meter  tests  performed 

V35 

Number  of  Logic  tests  performed 

V36 

Number  of  Current  tests  performed 

V37 

Number  of  Frequency  tests  performed 

V38 

Number  of  Continuity  tests  performed 

V39 

Number  of  Adjustment  tests  performed 

V40 

Final  Score  of  the  episode 

V41,  V42.  V43 

TBD  --  these  are  for  possible  future  expansion 

TBD  -  To  be  determined. 
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FIGURE  3.  MATRIX  OF  DATA  REG.  ED  FOR  HYPOTHESES  TESTING 
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Unit  (LRU)  incorrectly  and  it  was  not  the  fault. 

Total  number  of  same  teat  types  made  consecutively  at  the  same 
lest  point. 


RESULTS 


Demographic  Data 

Demographic  data  were  collected  for  53  students  and  13  instructors.  Due  to  missing  data,  five 
of  the  students  were  dropped  from  the  data  base  For  the  remaining  48  students,  the  average  time 
in  service  was  2.23  years.  The  data  for  two  of  the  instructors  were  dropped  because  they  did  not 
hold  NTC  ET-1453.  For  the  remaining  1 1  instructors,  9  had  a  rate  of  ETl  and  2  had  a  rate  of  ET2; 
the  average  paygrade  was  5.82.  The  average  time  in  service  for  instructors  was  10.41  years  and 
average  time  in  paygrade  was  3.64  years.  The  instructors  had  held  NEC  ET-1453  for  an  average  of 
4.67  years  and  had  worked  on  the  NAVMACS(V)/SATCOM  hardware  in  the  fleet  an  average  of 
2.94  years.  Ip  addition,  they  averaged  16.18  months  as  NAVMACS(V)/SATCOM  instructors.  For 
complete  listings  of  student  and  instructor  demographic  data,  see  Appendix  A.  See  Appendix  B  for 
complete  hypotheses  testing  data. 

Experience  (Hypotheses  1  and  3) 

Hypothesis  1  is  stated  as  follows:  Instructors  (experts)  will  score  significantly  higher  on  the 
TAE  test  than  students  (novices).  A  one  way  analysis  of  variance  (ANOVA)  wa.s  performed  to  test 
this  hypothesis.  As  shown  in  Table  4,  the  F  ratio  value  is  2.271  with  a  probability  of  1373,  which 
is  not  significant.  Thus,  the  instructors  did  not  score  significantly  higher  on  the  TAE  test  than 
students. 


Table  4 


ANOVA  for  Student  TAE  Final  Scores  vs.  Instructor  TAE  Final  Scores 


GROUP 

MEAN 

N 

Students 

70.396 

48 

Instructors 

73.422 

11 

GRAND  MEAN  70.960 

59 

SOURCE 

SUM  OF  SQUARES 

D.F. 

MFAN  SQUARE 

F  RATIO 

PROB 

BETWEEN 

81.973 

1 

81.973 

2.271 

.1373 

WITHIN 

2057.124 

57 

36.090 

TOTAL 

2139.098 

58 

Hypothesis  3  is  stated  as  follows:  Subjects  with  a  longer  time  in  the  electronics  rate  (i.e.,  TIS) 
will  score  Significantly  higher  on  the  TAE  test  than  subjects  with  less  time  in  that  rate.  As  shown 
in  Table  5,  the  correlation  between  time  in  electronics  rate  and  TAE  score  is  13.68,  which  is  not 
significant.  Therefore,  subjects  with  a  longer  time  in  the  electronics  rate  did  not  score  significantly 
higher  on  the  TAE  test  than  subjects  with  less  time  in  that  rate. 
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Table  5 


Correlation  Matrix:  TIS  vs.  TAE  Final  Scores 


(N  =  59) 

TIS 

TAE  SCORE 

TIS 

1.0(X)00 

TAE  SCORE 

.13676 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  .21638. 


Electronics  Knowledge  (Hypotheses  4  through  6) 

Hypothesis  4  is  stated  as  follows:  Students  (novices)  with  higher  academic  “C”  school  final 
scores  will  score  higher  on  the  TAE  test  than  students  (novices)  with  lower  scores.  As  shown  in 
Table  6,  the  correlation  between  academic  “C”  school  fmal  scores  (over  course  final  score)  and 
TAE  test  scores  is  .302,  which  is  significant  at  the  .05  significance  level.  Thus,  there  was  a 
significant  positive  correlation  between  academic  “C”  school  final  scores  and  TAE  test  scores.  On 
the  other  hand,  as  shown  in  Table  7,  the  correlation  between  academic  “C”  school  comprehensive 
scores  (final  test)  and  TAE  test  scores  is  .173,  which  is  a  positive  correlation  but  not  significant. 
Therefore,  academic  “C”  school  final  scores  were  significantly  correlated  with  TAE  test  scores,  but 
“C”  school  comprehensive  scores  were  not. 


Table  6 

Correlation  Matrix:  “C”  School  Final  Scores  vs.  TAE  Final  Scores 

(N  =  48) 


FINAL  SCORE 

TAESCX)RE 

nNAL  SCORE 

1.00000 

TAE  SCORE 

.30181 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  = 

.24045. 
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Table  7 


Correlation  Matrix:  “C”  School  Comprehensive  Scores  vs.  TAE  Final  Scores 

(N  =  48) 


COMPREHENSIVE  SCORE 

TAE  SCORE 

COMPREHENSIVE  SCORE 

1.{X)(X)0 

TAE  SCORE 

.17311 

l.CXXXX) 

CRITICAL  VALUE  (l-TAIL,  .05)  =  +/-  .24045. 


Hypothesis  5  is  stated  as  follows;  Students  (novices)  with  higher  academic  “C”  school 
subsystem  test  scores  will  score  higher  on  the  TAE  subsystem  tests  (episodes)  than  students 
(novices)  with  lower  “C”  school  subsystem  test  scores.  Each  subsystem  includes  the  following 
equipment;  Subsystem  1  =  UYK20;  Subsystem  2  =  USH26,  USQ69,  RD397,  TT624;  Subsystem 
3  =  CV3333;  Subsystem  4  =  ON143. 

Table  8  summarizes  the  results  of  the  correlational  analyses.  Complete  correlational  matrices 
for  each  subsystem  are  presented  in  Appendix  B.  For  Subsystem  1 ,  the  correlation  of  academic  “C” 
school  subsystem  test  scores  with  TAE  subsystem  test  scores  is  .277,  which  is  significant  at  the  .05 
level.  Subsystem  2  has  a  correlation  of .  176,  which  is  not  significant.  Both  Subsystems  3  and  4  have 
negative  correlations  of -.181  and  -.220  respectively,  which  are  not  significant.  Therefore,  the  only 
significant  correlation  between  academic  “C”  school  subsystem  test  scores  and  TAE  subsystem 
test  scores  was  for  Subsystem  1,  the  UYK20. 

Table  8 

Correlation  Between  “C”  School  Subsystem  Test  Scores  and  TAE  Subsystem  Test  Scores 

(N  =  48) 


PREDICTOR 

CORRELATION  WITH  SCHOOL  FINAL 

TAE  Average  Score  Subsystem  1 

.277  * 

TAE  Average  Score  Subsystem  2 

.176 

TAE  Average  Score  Subsystem  3 

-.181 

TAE  Average  Score  Subsystem  4 

-.220 

*p  <  .05. 


Hypothesis  6  is  stated  as  follows;  Students  with  higher  appropriate  ASVAB;  that  is,  El  and 
ET  selection  criteria  scores  (GS+EI+MK;  plus  AR  and  AFQT)  will  score  higher  on  the  TAE  test 
than  subjects  with  lower  ASVAB  and  selection  scores.  Table  9  summarizes  the  results  of  the 
correlational  analyses  between  ASVAB  selection  criteria  scores  and  TAE  test  scores.  Complete 
correlational  matrices  for  ASVAB  scores  are  presented  in  Appendix  B. 
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As  shown  in  Table  9,  all  but  one  of  the  correlations  is  negative.  The  only  significant 
correlation  between  ASVAB  scores  and  TAE  score  is  Arithmetic  Reasoning  (AR)  with  a  negative 
correlation  of  -.325,  significant  at  the  .05  level.  The  only  positive  correlation  is  between  General 
Science  (GS)  and  TAE  score,  with  a  non-significant  correlation  of  .1 15. 

Table  9 

Correlation  Between  ASVAB  Selection  Criteria  Scores  and  TAE  Test  Scores 

(N  =  48) 


ASVAB  SCORE 

CORRELATION  WITH  TAE  TEST  SCORES 

AFQT 

-.004 

GS 

.115 

AR 

-.325* 

MK 

-.101 

El 

-.067 

GS  -t-MK  +  El  (ASVAB  1) 

-.027 

GS  -t-  MK  -t-  El  (ASVAB  T) 

-.131 

*p  <  .05. 


Electronics  Performance  Proficiency  (Hypotheses  7  through  12) 

Hypothesis  7  is  stated  as  follows:  Subjects  with  a  higher  level  of  troubleshooting  proficiency 
will  make  fewer  invalid  checks  than  less  proficient  subjects.  As  shown  in  Table  10,  the  correlation 
between  TAE  score  and  the  number  of  invalid  checks  is  -.171,  which  is  not  significant.  Therefore, 
subjects  with  higher  troubleshooting  proficiency  tended  to  make  fewer  invalid  checks  than  less 
proficient  subjects. 


Table  10 

Correlation  Matrix:  TAE  Score  vs.  Invalid  Checks 
(N  =  59) 


TAE  SCORE 

INVALID  CHECKS 

TAE  SCORE 

1.00000 

INVALID  CHECKS 

-.17107 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  -k/-  .21638. 


Hypothesis  8  is  stated  as  follows:  Subjects  with  a  higher  level  of  troubleshooting  proficiency 
will  make  fewer  illogical  approaches  than  less  proficient  subjects.  As  shown  ui  Table  11,  the 
correlation  between  TAE  score  and  the  number  of  illogical  approaches  is  -.341,  which  is  significant 
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at  the  .01  level.  Therefore,  subjects  with  higher  troubleshooting  proficiency  made  significantly 
fewer  illogical  approaches  than  less  proficient  subjects. 


Table  11 

Correlation  Matrix:  TAE  Score  v&  Illogical  Approaches 

(N  =  S9) 


TAE  SCORE 

ILLOGICAL  APPROACHES 

TAE  SCORE 

1.00000 

ILLOGICAL  APPROACHES 

-.34057 

1.00000 

CRITICAL  VALITE  (1-TAIL,  .05)  =  +/-  .21638. 


Hypothesis  9  is  stated  as  follows:  Subjects  with  a  higher  level  of  troubleshooting  proficiency 
will  make  fewer  incorrect  solutions  than  less  proficient  subjects.  As  shown  in  Table  12,  the 
correlation  between  the  TAE  score  and  the  number  of  incorrect  solutions  is  -.697,  which  is 
significant  at  the  .001  level.  Thus,  subjects  with  higher  troubleshooting  proficiency  made 
significantly  fewer  incorrect  solutions  than  less  proficient  subjects. 

Table  12 

Correlation  Matrix:  TAE  Score  vs.  Incorrect  Solutions 

(N  =  59) 


TAE  SCORE 

INCORRECT  SOLUTIONS 

TAE  SCORE 

1.00000 

INCORRECT  SOLUTIONS 

-.69676 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  +/-  .21638 


Hypothesis  10  is  stated  as  follows;  Subjects  with  a  higher  level  of  troubleshooting 
proficiency  will  make  fewer  redundant  checks  than  less  proficient  subjects.  As  shown  in  Table  13, 
the  correlation  between  TAE  score  and  the  number  of  redundant  checks  is  -.085,  which  is  not 
significant.  Therefore,  subjects  with  higher  troubleshooting  proficiency  tended  to  make  fewer 
redundant  checks  than  less  proficient  subjects. 
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Table  13 


Correlation  Matrix:  TAE  Score  vs.  Redundant  Checks 

(N  =  59) 


TAE  SCORE 

REDUNDANT  CHECKS 

TAE  SCOPE 

1.00000 

REDUNDANT  CHECKS 

-.08543 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05) 

=  -h/-.21638. 

Hypothesis  11  is  stated  as  follows:  Subjects  with  a  higher  level  of  troubleshooting 
proficiency  will  test  significantly  more  proof  points  than  less  proficient  subjects.  As  shown  in 
Table  14,  the  correlation  between  the  TAE  score  and  the  number  of  proof  points  is  .561,  which  is 
significant  at  the  .001  level.  Therefore,  subjects  with  higher  troubleshooting  proficiency  tested 
significantly  more  proof  points  than  less  proficient  subjects. 

Table  14 

Correlation  Matrix:  TAE  Score  vs.  Proof  Points 
(N  =  59) 


TAE  SCORE 

PROOF  POINTS 

TAE  SCORE 

1.00000 

PROOF  POINTS 

.56097 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  -t-/-  .21638. 


Hypothesis  12  is  stated  as  follows:  In  general,  subjects  with  a  higher  level  of  troubleshooting 
proficiency  will  make  significantly  fewer  tests  than  less  proficient  subjects.  As  shown  in  Table  15, 
the  correlation  between  the  level  of  troubleshooting  proficiency  and  number  of  tests  is  -.552,  which 
is  significant  at  the  .001  level.  Therefore,  subjects  who  demonstrated  a  higher  level  of 
troubleshooting  proficiency  through  higher  TAE  scores,  made  significantly  fewer  tests  than  less 
proficient  subjects. 
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Table  15 


Correlation  Matrix:  TAE  Score  vs.  Number  of  Tests 

(N  =  59) 


TAE  SCORE 

NUMBER  TESTS 

TAE  SC(JkE 

1.00000 

NUMBER  TESTS 

-.55201 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  +!-  .21638. 


Difficulty  Level  (Hypotheses  13  through  16) 

Hypothesis  13  is  stated  as  follows:  The  more  difficult  the  episode,  the  longer  tne  average  time 
needed  to  find  the  solution  across  subjects.  As  shown  in  Table  16,  the  correlation  of  TAE  difficulty 
with  length  of  time  to  find  the  solution  is  .931,  which  is  significant  at  the  .001  level.  Therefore,  the 
more  difficult  the  episode  the  longer  subjects  took  to  find  the  solution. 

Table  16 

Correlation  Matrix:  TAE  Difficulty  vs.  Time 
(N  =  14) 


AVERAGE  TIME 

Z SCORE 

AVERAGE  TIME 

1.00000 

Z  SCORE 

.93051 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  -t-/-  .45900 

Hypothesis  14  is  stated  as  follows;  On  episodes  of  equal  difficulty,  subjects  with  a  higher 
level  of  troubleshooting  proficiency  will  take  significantly  less  time  than  less  proficient  subjects  in 
finding  the  solution.  The  episodes  were  assigned  (based  on  their  Z  scores)  to  difficulty  levels  as 
follows  with  level  1  being  the  easiest  and  level  5  the  most  difficult:  (1)  episodes  7,  8,  (2)  episodes 
1, 2,  3, 12,  (3)  episodes  4, 1 1, 14,  (4)  episodes  5, 13,  and  (5)  episodes  6, 9, 10.  Table  17  summarizes 
the  results  of  the  correlational  analysis  between  difficulty  levels  and  time.  Complete  correlational 
matrices  for  each  level  are  presented  in  Appendix  B.  As  shown  in  Table  17,  Hypothesis  14  was 
sufxponcd  for  each  level.  Therefore,  for  each  of  the  difficulty  levels,  subjects  with  a  higher  level  of 
troubleshooting  proficiency  took  significantly  less  time  than  less  proficient  subjects  in  finding  the 
soluuon. 
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Table  17 


Correlation  Between  Difficulty  Level  and  Time  to  Solution 

(N  -  59) 


DIFRCLXTY  LEVEL 

CORRELATION  WITH  TIME  TO  SOLUTION 

Level  1 

-.813  * 

Level  2 

-.336  ** 

Level  3 

-.747  * 

Level  4 

-.736* 

Level  5 

-.588* 

*p<.001. 

**p<.01. 


Hypothesis  15  is  stated  as  follows:  The  more  difficult  the  episode,  the  less  time  the  instructors 
will  take  to  find  the  TAE  test  solutions  when  compared  to  the  students  (novices).  As  shown  in 
Table  18,  the  difficulty  level  of  the  episode  and  the  difference  in  time  between  instructors  and 
students  to  find  TAE  test  solutions  is  negatively  correlated  -.347,  which  is  not  significant. 
Although  a  significant  difference  was  not  found,  the  more  difficult  the  episode,  the  less  time 
instructors  tended  to  take  to  find  the  TAE  test  solutions  when  compared  to  the  students. 

Table  18 

Correlation  Matrix:  TAE  Difficulty  vs.  Time  Differences 
Between  Students  and  Instructors 
(N  =  14) 


Z  SCORE 

TIME  DIFFERENCE 

Z  SCORE 

1.00000 

TIME  DIFFERENCE 

-.34656 

1.00000 

CRITICAL  VALUE  (1-TAIL,  .05)  =  +/-  .45900. 


Time  (Hypotheses  17  and  18) 

Hypothesis  17  is  stated  as  follows:  Subjects  with  a  higher  level  of  troubleshooting 
proficiency  will  take  significantly  less  total  time  to  find  TAE  episode  solutions  than  less  proficient 
subjects.  As  shown  in  Table  19,  the  correlation  between  TAE  score  and  total  time  to  find  episode 
fault  is  -.492,  which  is  significant  at  the  .001  level.  Thus,  subjects  with  a  higher  level  of 
troubleshooting  proficiency  took  significantly  less  time  to  find  episode  solutions  than  less 
proficient  subjects. 
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Table  19 


Correlation  Matrix:  TAE  Score  vs.  Time  to  Solution 

(N  =  59) 


TAE  SCORE 

TIME 

TAE  SCORE 

l.(XX)00 

TIME 

-.49233 

1.00000 

CRITICAL  VALUE  (I -TAIL,  .05)  =  +/-  .21638. 

Hypothesis  18  is  stated  as  follows:  In  general,  subjects  with  the  higher  level  of 
troubleshooting  proficiency  will  take  a  significantly  longer  time  than  less  proficient  subjects  before 
making  the  first  test  point.  As  shown  in  Table  20,  the  correlation  between  TAE  score  and  time  to 
first  test  point  is  -.238,  which  is  significant  at  the  .05  level.  Consequently,  subjects  with  a  higher 
level  of  troubleshooting  proficiency  took  a  significantly  longer  time  before  making  the  first  test 
point  than  less  proficient  subjects. 


Table  20 

Correlation  Matrix:  TAE  Score  vs.  Time  to  First  Test  Point 

(N  =  59) 


TAE  SCORE 

TIME  TILL 

TAE  SCORE 

1.00000 

TIME  TILL 

-.23814 

1.00000 

CRmCAL  VALUE  (1-TAIL,  .05)  =  -t-/-  .21638. 

Complex  Test  Equipment  (Hypothesis  19) 

Hypothesis  19  is  stated  as  follows:  Subjects  with  a  higher  level  of  troubleshooting 
proficiency  will  make  significantly  more  tests  using  an  oscilloscope  than  less  proficient  subjects. 
As  shown  in  Table  21,  the  correlation  between  TAE  score  and  the  number  of  oscilloscope  tests  is 
.  1 68,  which  is  not  significant.  Thus,  subjects  with  higher  TAE  scores  did  not  make  significantly 
more  tcst^  using  an  oscilloscope  than  less  proficient  subjects. 
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Table  21 


Correlation  Matrix:  TAE  Score  vs.  Oscilloscope  Tests 

(N  =  59) 


_ TAE  SCORE _ OSCILLOSCOPE 

TAE  SCORE  1.00000 

OSCILLOSOOPE _ .16771 _ 1,00000 _ 

CRITICAL  VALUE  (1-TAIL,  .05)  =  +/-  .21638. 

Ranking  (Hypotheses  20  through  22) 

Hypothesis  20  is  stated  as  follows:  The  higher  the  student’s  TAE  class  rank,  the  higher  the 
student  will  be  ranked  in  terms  of  troubleshooting  proficiency  by  instructors/work  center 
supervisors.  The  correlation  values  for  TAE  class  ranking  versus  ranking  by  instructor/work  center 
supervisor  are  presented  in  Table  22.  Complete  correlational  matrices  for  each  class  (by  graduation 
date)  are  presented  in  Appendix  B. 


Table  22 

Correlation  Between  TAE  Class  Rank  and  Instructor  Rankings 


CLASS 

GRADUATION  DATE 

CORRELATION 

89030 

14  APR  89 

.964* 

89040 

12  MAY  89 

.357 

89060 

09JUN89 

.464 

89070 

07  JUL89 

.486 

89080 

04  AUG  89 

-.143 

89100 

25  AUG  89 

-.071 

89120 

29  SEP  89 

.964* 

*p<.001. 


Hypothesis  20  was  supported  for  classes  89030  and  89120  at  the  .001  level  of  significance. 
The  corrcliinon  between  TAE  class  ranking  and  instructor/work  center  supervisor  ranking  was  not 
significant  for  any  of  the  other  classes.  Although  not  significant,  classes  89080  and  89100  actually 
indicated  an  inverse  relationship. 

Hypothesis  21  is  stated  as  follows:  The  higher  the  student’s  TAE  class  rank,  the  higher  the 
student  will  be  ranked  in  the  class.  The  correlation  values  for  student  TAE  class  ranking  versus 
student  ranking  in  “C”  school  classes  are  presented  in  Table  23.  Complete  correlational  matrices 
for  each  class  (by  graduation  date)  are  presented  in  Appendix  B. 
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Table  23 


Correlation  Between  TAE  Class  Rank  and  “C” 

School  Class  Rank 

CLASS 

GRADUATION  DATE 

CORRELATION 

89030 

14  APR  89 

.893* 

89040 

12  MAY  89 

.571 

89060 

09JUN  89 

-.143 

89070 

07  JUL89 

.486 

89080 

04  AUG  89 

.371 

89100 

25  AUG  89 

-.595 

89120 

29  SEP  89 

.607 

*p  <  .01. 

Hypothesis  21  was  supported  for  class  89030  at  the  .01  level  of  significance.  For  the  other 

classes,  the  correlation  between  TAE  class  ranking  and  ranking  in  “C”  school  class  was  not 
significant.  Although  not  significant,  classes  89040  and  89120  indicated  a  strong  positive 

correlation.  Conversely,  class  89100  showed  a  strong  inverse 

relationship  between  TAE  class 

ranking  and 

“C”  school  class  ranking. 

Hypothesis  22  is  stated  as  follows:  The  higher  the  instructor  ranking  of  the  student  in  terms 

of  troubleshooting  proficiency,  the  higher  will  be  the  student’s  ranking  in  the  class.  The  correlation 

values  for  instructor  ranking  of  students  versus  student  ranking 

in  “C”  school  class  are  presented 

in  Table  24.  Complete  correlational  matrices  for  each  class  (by  graduation  date)  are  presented  in 

Appendix  B 

Table  24 

Correlation  Between  “C”  School  Class  Rank  and  Instructor  Ranking 

CLASS 

GRADUATION  DATE 

CORRELATION 

89030 

14  APR  89 

.964* 

89040 

12  MAY  89 

-.024 

89060 

09  JUN89 

-.357 

89070 

07JUL89 

.750** 

89080 

04  AUG  89 

.333 

89100 

25  AUG  89 

.633** 

89120 

29  SEP  89 

.643 

♦p<.001. 

**p  <  m. 
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Hypothesis  22  was  supported  for  classes  89030,  89070,  and  89100.  The  strongest  correlation 
was  for  class  89030  (.001  level)  followed  by  classes  89070  and  89100  (both  at  .05  level).  Although 
not  significant,  class  89120  showed  a  strong  positive  correlation  between  instructor  student 
ranking  and  class  student  ranking.  For  the  other  classes,  89080  showed  a  weaker  positive 
correlation  and  classes  89040  and  89060  indicated  an  inverse  relationship. 

CONCLUSIONS 


Experience 

The  results  of  the  analyses  of  hypotheses  1  and  3  indicate  that  there  was  no  significant 
relationship  between  experience  and  TAE  performance.  Hypothesis  2  was  dropped  due  to  the  lack 
of  data  for  experienced  fleet  personnel: 

1 .  Instructors  (experts)  will  score  significantly  higher  on  the  TAE  test  than  students  (novices). 

3.  Subjects  with  a  longer  time  in  the  electronics  rate  (i.e.,  TIS)  will  score  significantly  higher  on 
the  TAE  test  than  subjects  with  less  time  in  that  rate. 

Several  explanations  might  address  the  apparent  anomaly  that  there  was  no  significant 
difference  in  the  performance  of  the  students  and  instructors.  Instructors  in  the  course  have  no 
requirement  to  have  or  maintain  system  qualification.  Whereas,  students  must  prove  their  system 
qualification  or  they  cannot  graduate  or  be  awarded  the  NEC.  It  may,  however,  be  argued  that  the 
evaluation  technique  is  suspect  and  that  these  results  question  the  validity  of  the  TAE  approach. 

Electronics  Knowledge 

The  results  of  the  analyses  of  hypotheses  4  through  6  indicate  that  there  was  no  generally 
consistent  relationship  between  electronics  knowledge  and  TAE  performance: 

4.  Students  (novices)  with  higher  academic  “C”  school  final  scores  will  score  higher  on  the 
TAE  test  than  students  (novices)  with  lower  scores. 

5.  Students  (novices)  with  higher  academic  “C”  school  subsystem  test  scores  will  score  higher 
on  the  TAE  subsystem  tests  (episodes)  than  students  (novices)  with  lower  “C”  school  sub¬ 
system  test  scores. 

6.  Students  with  higher  appropriate  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 
(i.e.,  El  [Electronics  Information]  and  ET  [Electronics  Technician]  selection  criteria  scores 
[GS+EI+MK;  plus  AR  and  AFQT])  will  score  higher  on  the  TAE  test  than  subjects  with  low¬ 
er  A’^VAB  and  selection  scores. 

Hypothesis  4  showed  a  relationship  in  which  performance  testing  was  a  component  of  the 
academic  school  final  score  used.  There  was,  however,  a  negative  relationship  between  the  scores 
used  to  determine  selection  to  the  occupational  speciality  and  the  performance  scores.  Of  special 
note  is  the  significant  negative  correlation  of  AR  to  TAE. 
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Electronics  Performance  Proficiency 

As  expected,  the  results  of  testing  hypotheses  7  through  12  showed,  generally,  a  consistent 
significant  negative  correlation  between  the  hypothesis  proficiency  factors  and  TAE  j>erformance: 

7.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  invalid  checks 
than  less  proficient  subjects. 

8.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  illogical  ap¬ 
proaches  than  less  proficient  subjects. 

9.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  incorrect  solu¬ 
tions  than  less  proficient  subjects. 

10.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  fewer  redundant  checks 
than  less  proficient  subjects. 

11.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  test  significantly  more  proof 
points  than  less  proficient  subjects. 

12  In  general,  subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  significantly 
fewer  tests  than  less  proficient  subjects. 

The  only  factor  that  failed  to  show  significance  was  redundant  checks.  This  may  have  been 
a  result  of  the  design  of  the  delivery  system  (based  on  the  desigri  of  the  operational  hardware)  and/ 
or  the  method  of  determining  redundancy,  which  was  seriously  restricted  due  to  programming 
difficulty.  As  a  general  conclusion,  the  results  of  this  set  of  hypotheses  strongly  support  the  validity 
of  the  TAE  technique  and  approach. 

Difficulty  Level 

The  results  of  testing  Hypotheses  13  through  15  indicate  that  there  was  a  general  and 
consistent  significant  relationship  between  the  difficulty  of  a  troubleshooting  episode  and  TAE 
performance.  Hypothesis  16  was  dropped  due  to  the  lack  of  data  for  experienced  fleet  personnel. 
Difficulty  was  defined  as  follows  by  each  hypothesis: 

13.  The  more  difficult  the  episode,  the  longer  the  average  time  to  find  the  solution  across  sub¬ 
jects. 

14  On  episodes  of  equal  difficulty,  subjects  with  a  higher  level  of  troubleshooting  proficiency 
will  take  significantly  less  time  than  less  proficient  subjects  in  finding  the  solution. 

15.  The  more  difficult  the  episode,  the  less  time  the  instructors  will  take  to  find  the  TAE  test  so¬ 
lutions  when  compared  to  the  students  (novices). 

Generally  the  results  regarding  difficulty  were  as  expected  (i.e.,  more  difficult,  more  time). 
At  different  levels  of  difficulty  better  performers  took  less  time.  An  unexpected  result  was  the  lack 
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of  significant  difference  between  students  and  instructors.  The  difference  was,  however,  strongly 
in  the  direction  expected. 

Time 


As  expected,  there  was  a  general  and  consistent  significant  relationship  between  time  and  the 
TAE  episode  results: 

17.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  take  significantly  less  total 
time  to  find  TAE  episode  solutions  than  less  proficient  subjects. 

1 8  In  general,  subjects  with  the  higher  level  of  troubleshooting  proficiency  will  take  a  signifi¬ 
cantly  longer  time  than  less  proficient  subjects  before  making  the  first  test  point. 

The  results  of  hypothesis  18  may  indicate  that  an  in-depth  investigation  of  the  behavior  and 
cognitive  protocols  could  result  in  a  dramatic  change  in  the  way  the  training  community  presents 
troubleshooting  training. 

Complex  Test  Equipment 

The  results  of  the  analysis  indicates  that  there  was  no  significant  relationship  between  TAE 
performance  and  the  use  of  complex  test  equipment: 

19.  Subjects  with  a  higher  level  of  troubleshooting  proficiency  will  make  significantly  more  tests 
using  an  oscilloscope  than  less  proficient  subjects. 

Given  the  nature  of  the  hardware  system  and  the  resulting  TAE  delivery  system  (as  dictated 
by  the  maintenance  philosophy  of  the  hardware),  there  did  not  appear  to  be  sufficient  opportunity 
for  the  subjects  to  exercise  use  of  complex  test  equipment  in  the  TAE  episodes.  Therefore,  the  fact 
that  there  was  no  statistically  significant  result  may  have  had  no  practical  meaning  due  to  the  TAE 
design  restrictions. 

Ranking 

Ranking  was  defined  by  the  hypotheses  as  follows: 

20  The  higher  the  student’s  TAE  class  rank,  the  higher  the  student  will  be  ranked  in  terms  of 
troubleshooting  proficiency  by  instructors/woric  center  supervisors. 

21  The  higher  the  student’s  TAE  class  rank,  the  higher  the  student’s  ranking  in  the  class. 

22  The  higher  the  instructor  ranking  of  the  student  in  terms  of  troubleshooting  proficiency,  the 
higher  the  student’s  ranking  in  the  class. 

There  were  no  consistent  results  in  rankings  across  instructors,  TAE  performance,  or  school 
performance.  In  fact,  in  several  classes,  inverse  relationships  were  shown.  There  were,  however, 
some  classes  (particularly  89030)  where  there  was  a  consistent  significant  relationship  indicated 
across  all  three  hypotheses. 
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FUTURE  EFFORTS 


The  following  recommendations  for  future  efforts  address  issues  related  to  the  results  of  the 
TAE  test  and  evaluation.  The  recommendations  are  directed  towards  validity  and  reliability 
questions,  as  well  as  the  modification  and  improvement  of  the  TAE  approach  to  provide  a 
troubleshooting  assessment  and  enhancement  capability  to  the  fleet  and  Navy  training  community. 

1 .  Further  investigate  TAE  as  related  to  validity  and  reliability.  A  number  of  the  hypotheses  ad¬ 
dress  the  issues  of  validity  and  reliability.  The  method  of  design  and  development  of  the  TAE 
approach  and  delivery  system  strongly  supports  the  face  validity  of  TAE.  Subject  matter  ex¬ 
perts  were  involved  in  all  phases  of  the  project.  They  determined  the  factors  of  evaluation; 
deiermined  the  weights  of  the  factors;  determined  the  evaluation  scheme;  determined  the 
troubleshooting  episodes  to  be  used;  developed  the  troubleshooting  episodes  and  participated 
in  the  test  and  evaluation.  Since  the  test  and  evaluation  resii't*^  ^re  ^omewhat  ambiguous,  fur¬ 
ther  investigation,  particularly  in  those  areas  that  deal  with  validity  and  reliability,  should  be 
conducted. 

2.  Further  investigate  TAE  as  related  to  experience  and  troubleshooting  performance.  The  lack 
of  a  significant  relationship  between  experience,  as  defined  in  this  study,  and  troubleshooting 
performance,  as  measured  by  the  TAE  approach,  causes  one  to  consider  whether  the  experi¬ 
ence  measures  were  appropriate,  whether  an  appropriate  set  of  subjects  was  tested,  and 
whether  the  TAE  delivery  and  evaluation  systems  is  valid.  Given  the  face  validity  of  the  TAE 
approach  and  the  high  level  of  expectation  by  subject  matter  experts  of  the  relationship  be¬ 
tween  experience  and  performance,  it  seems  that  further  testing  should  be  performed  to  re¬ 
solve  the  apparent  incongruity  between  these  two  measures. 

3.  Further  investigate  TAE  as  related  to  academic  and  knowledge  factors.  As  with  the  results  of 
a  number  of  other  studies  of  this  type,  there  was  no  consistent  relationship  between  knowl¬ 
edge  of  theory  and  ability  to  perform.  This  may  have  been  related,  in  part,  to  the  method  of 
determining  knowledge  as  academic  success  in  the  school.  The  method  of  testing  in  the 
school  does  not  appear  to  provide  discriminatory  capability  and,  therefore,  correlational  anal¬ 
yses  do  not  show  statistically  significant  results.  The  schools  need  testing  methods  and  tech¬ 
niques  that  provide  for  a  true  way  to  discriminate  between  student’s  academic  and 
performance  ability  and  a  more  structured,  formalized  and  objective  way  to  assess  student  be¬ 
haviors.  Otherwise,  the  effects  of  a  change  to  instructional  methods  or  techniques  cannot  be 
assessed  in  terms  of  relative  value  to  the  course  outcomes.  Then,  further  TAE  testing  could 
be  accomplished  to  determine  the  resulting  relationships. 

4.  Further  investigate  the  relationships  between  selection  requirements  and  troubleshooting  per¬ 
formance.  In  the  short  term,  questions  arise  as  a  result  of  the  failure  of  the  performance  results 
to  positively  relate  to  the  ASVAB  tests  used  to  select  personnel  for  this  occupational  speci¬ 
ality.  In  fact,  a  consistent  negative  trend  brings  the  entire  screening  and  selection  process  into 
question.  This  would  seem  to  indicate  that,  while  the  ASVAB  tests  may  relate  to  academic 
performance,  there  may  be  no  relationship  between  ASVAB  performance,  TAE  perfor¬ 
mance,  and/or  on-the-job  performance. 
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I  iinlicr  anal>  /e  the  TAfi  data  and  results  to  improve  the  discnminaicry  and  predicii\  e  capa- 
bilit_\  of  the  TAE  approach.  The  results  of  perfonnance  of  the  subjects  on  the  TAE  episodes 
should  be  subjected  to  behavioral  protocol  analyses  to  develop  a  model  of  troubleshootinj: 
and  further  analyses  of  good  vs.  bad  troubleshooters.  Once  behavioral  models  are  construct¬ 
ed.  further  cognitive  protocol  analysses  could  be  perceived. 

tv  I  I  'lher  test  the  TAE  approach  on  other  subjects  and  on  other  equipment  and  equipment 
type  T^ie  T.AE  approach  should  be  further  investigated  on  hardware  that  allows  wider  and 
les'-  restrictive  utilization  of  test  equipment.  It  may  also  be  possible  to  select  specific  trouble- 
shiKsting  episodes  that  provide  wider  utilization  of  test  equipment  types.  If  so.  it  is  recom- 
mend^-d  that  this  type  of  investigation  take  place  to  determine  if  certain  episodes  and 
harduaie  types  require  special  test  equipment  use  capability. 

Also,  It  would  be  hazardous  to  draw  major  and  sweeping  conclusions  regarding  the  efficacs 
of  the  current  T.AE  system.  It  would  be  advisable  to  irive.siigate  this  approach  to  other  high-tech 
hardware  s\  stems  as  well  as  other  occupational  areas  (i.e.,  mechanical  hardware  troubleshooters' 
repair  personnel).  It  is  recommended  that  a  TAE  type  delivery  system  be  developed  fc>r  a  number 
('f  other  high  and  mid-tech  hardware  systems. 

Develop  additional  troubleshooting  epi.sodes  to  provide  directive  training,  guided  training, 
and  tests  w  ith  feedback.  Then,  a  complete  and  comprehensive  troubleshooting  skill  develop¬ 
ment.  maintenance,  assessment,  and  evaluation  program  would  be  available  for  personnel 
from  the  novice  to  expert  skill  levels.  It  could  be  used  for  active  duty  personnel  in  the  school 
or  fleet  en\  ironment  and  for  re.serve  personnel  at  the  readiness  centers  or  aboard  ship  during 
active  duty  periods. 
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80) .  0)0) 

84 . 0)0) 

C* 

96  .O'O 

94  . 00 

93.30 

1 0)0) .  0)0) 

95 . 0)0) 

1 0)0) .  O'O') 

88 .00) 

6 

96 .00’ 

90  .  0i7 

86 . 70 

80) .  O'O) 

90)  .0)0) 

90) .  0)0) 

86 . 0)0' 

7 

88 .00 

84  .00 

86 . 70 

88 .0)0) 

90 . 0)0) 

100).  0)0) 

88  .00 

8 

92  . 0(') 

94  .00 

86.70 

84  .0)0 

95  .0)0) 

90) .  0)0 

86  .00 

FINAL  SC 

CLASS  ST 

INST  RNK 

ASVAB  1 

ASVAB  T 

1 

91  .87 

7  .00 

7.00 

.00 

.00 

2 

89 . 00 

8  .00 

8  .00 

.00 

.  00 

3 

96.79 

3  .00 

2.00 

193.00 

251  .0)0 

4 

94  .62 

5.00 

5.00 

180.00 

242.00 

C*! 

98 .07 

1  .00 

6  .  00 

203.00 

265.00 

6 

96.96 

2 . 00 

4  .00 

1 82 . 00 

246.00 

7 

94  .77 

4  .00 

3.00 

196.00 

262.00 

8 

94  .04 

6.00 

1  .00 

181 .00 

244.00 

+  Indicates  students  dropped  for  missing  data 


A- 7 


demographic  data  FDR  CLASS  B9100  -  8/25/89  -  FILE:  TAEDS100 


HEADER 

DATA  FOR 

:  C:TAEDS100 

LABEL 

:  Demographics  -for 

Class  S89100 

NUMBER 

OF  CASES 

:  9  NUMBER 

OF  VARIABLES:  19 

SSN 

TIS 

AFOT 

GS 

AR 

MK 

El 

+  1 ♦****♦♦**♦ 

5.64 

.00 

.00 

.00 

.00 

.00 

2********** 

1  .29 

93  .00 

63  .00 

64  .00 

64  . 00 

64.00 

3*«-******** 

1  .48 

82.00 

65.00 

58.00 

58  .00 

62.00 

4********** 

1  .54 

80.00 

63.00 

62  .00 

55 . 00 

64  .00 

5***- *■****♦•)( 

1  .37 

98  .00 

65.00 

63 . 0(') 

60  .00 

69  .00 

6********** 

1  .41 

75.00 

65 . 0>0 

62  .00 

61  .00 

67.00 

1  .40 

51  .00 

44  .00 

55 . 00 

55 . 00 

51  .00 

6********** 

1  .35 

82.00 

62.00 

60  .00 

6  1  .  00 

62 . 00 

2.50 

87 .00 

64 . 00 

66  .00 

68.00 

54  . 00 

FINAL  1 

FINAL  2  FINAL  3 

FINAL  4 

FINAL  5 

FINAL  6 

COMP 

1 

88  .  (')(') 

84  .00 

86.70 

80 .00 

90  .00 

90 .00 

90 .00 

86  .  w(;) 

94  .00 

93.30 

96  .00 

95 . 0(:' 

90 . 00 

84  .00 

3 

88 

92 .00 

1  0('i  .01:1 

88  .00 

90 . 00 

80 .00 

88 .00 

4 

94  .(>)(;) 

98.00 

93.30 

92  .00 

95 . 00 

1  00 . 00 

88.00 

CT 

98 .00 

94  .00 

93.30 

88.00 

95 . 0(') 

1  00 . 01;) 

90  .  (;)0 

6 

96  .Oiji 

92 .00 

86.70 

88 . 00 

95 .00 

90  .00 

84 . 00 

7 

92 . 00 

84 . 00 

83 .30 

84 . 00 

95  .0(;i 

90 .00 

90 .00 

8 

78 .00 

88 .00 

86.70 

84 . 00 

85 . 0<’'> 

80 .00 

92.00 

9 

98  .00 

96.00 

93.30 

96.00 

95.00 

80  .00 

94  .00 

FINAL  SC 

CLASS  ST  INST  RNK 

ASVAB  1 

ASVAB  T 

1 

94  .29 

6 . 00 

7.00 

.00 

.00 

2 

96.93 

3.00 

6 . 00 

191 .00 

255 . 00 

3 

92.98 

9  .00 

8  .  00 

185.00 

243  .00 

4 

98.23 

1  .00 

4  .00 

182.00 

244  .00 

5 

97.93 

2  .00 

3  .00 

194 .00 

257.00 

6 

93.43 

7.00 

5 .00 

193.00 

255 .00 

7 

96.77 

5  .00 

2  .00 

150  .€>0 

205  .00 

8 

93.09 

8  .00 

9 . 00 

185.00 

245 .00 

9 

96  .83 

4  .00 

1  .00 

186.00 

252  .00 

+  Indlcfltep  ^.udents  dropped  for  missing  data 


DEMOGRAPHIC  DATA  FOR  CLASS  89120  -  9/29/89  -  FILE;  TAEDS120 


HEADER  DATA  FOR:  C:TAEDS120  LABEL:  Demographics  -for  Class  S89120 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  19 


SSN 

TIS 

AFQT 

GS 

AR 

MK 

El 

1********** 

2.78 

75.00 

62.00 

50  .00 

58.00 

58.00 

********** 

1  .78 

84  .00 

58.00 

60.00 

64.00 

67.00 

****** 

2.29 

96  .00 

58.00 

64  .00 

61  .00 

64  .00 

4*'*****  **** 

1  .68 

88.00 

64  .00 

61  .00 

65  .00 

51  .00 

********** 

3.79 

73.00 

53.00 

56.00 

52.00 

67.00 

fc********** 

1  .75 

93.00 

60 . 00 

62  .00 

66  .00 

55 . 00 

7********** 

1  .79 

40.00 

53.00 

51  .00 

60  .00 

67.00 

FINAL  1 

FINAL  2 

FINAL  3 

FINAL  4 

FINAL  5 

FINAL  6 

COMP 

1 

86 . 00 

96 . 00 

73.30 

96.00 

80  .00 

100 .00 

86 . 00 

96  .00 

92.00 

93.30 

SB .  00 

95  .00 

90  .00 

90 . 00 

3 

94  .00 

92 . 00 

1 00 .00 

92 . 00 

90 . 00 

90  .00 

82 . 00 

A 

96 .00 

98 . 0(') 

86 . 70 

84  .00 

90 .00 

1 0(') .  00 

84 . 00 

er 

80  .00 

90  .  <;i0 

1 00 . 00 

92.00 

100 .00 

100. 00 

86  .00 

6 

94  .00 

88 .00 

1 00 . 0!.'. 

96 . 0<i! 

85 . 00 

1 00 . 00 

96 . 00 

7 

84 . 0<i> 

86.00 

86.70 

80 .00 

1  •')(;) .  (')<') 

1 00 . 00 

84  .00 

FINAL  SC 

CLASS  ST 

INST  RNK 

ASVAB  1 

ASVAB  T 

1 

92.26 

7.00 

4  .00 

178.00 

228.00 

2 

96  .50 

3  .00 

2.00 

189 .00 

249  .00 

3 

96.12 

4.00 

7.00 

183.00 

247.00 

4 

96.56 

2.00 

3.00 

180 .00 

241  .00 

e* 

95.75 

5  .00 

5  .00 

172.00 

228.00 

6 

97.99 

1  .00 

1  .00 

181 .00 

243.00 

7 

95.39 

6  .00 

6  .00 

180 .00 

231 .00 

A-9 


INSTRUCTOR  DEMOGRAPHIC  DATA 


A-10 


DEMOGRAPHIC  DATA  FOR  INSTRUCTORS  -  FILE:  TAEDINST 


HEADER  DATA  FOR:  C:TAEtiINST  LABEL:  Demographics  -for  Instructors 
NUMBER  OF  CASES:  13  NUMBER  OF  VARIABLES:  1 


1 

TIS 
9 . 00 

12.75 

3 

15  .00 

4 

8.75 

+  T 

6  .50 

t 

£>.58 

-> 

3  3.83 

B 

7.42 

9 

8.92 

+  1('> 

7.67 

1  1 

7.50 

12 

6.92 

13 

17.83 

+  Indicates  instructors  did  not  hold  NEC  ET-1A53 


A-11 


TIME  IN  SERVICE  FOR  INSTRUCTORS 


HEADER  DATA  FOR;  C:TIS-INST  LABEL;  TIME  IN  SERVICE  FOR  INSTRUCTORS 
NUMBER  OF  CASES:  11  NUMBER  OF  VARIABLES;  1 


TIS 

12.75 

15 

8.75 

6.58 

13.83 
7.42 

8.92 
7 . 50 

6.92 

17.83 


-  DESCRIPTIVE  STATISTICS  - 

HEADER  DATA  FOR;  C;TIS-INST  LABEL;  TIME  IN  SERVICE  FOR  INSTRUCTORS 
NUMBER  OF  CASES:  11  NUMBER  OF  VARIABLES;  1 

NO.  NAME  N  MEAN  STD.  DEV.  MINIMUM  MAXIMUM 

1  TIS  11  10.4091  3.8015  6.5800  17.8300 


4 

r- 

6 

/ 

6 
9 
10 
1  1 


A-12 


INSTRUCTOR  DEMOGRAPHIC  DATA 


TIS 
9.00 
12  .75 
15.00 
8 .75 
6.58 

13.83 
7.42 

8.92 
7.50 

6.92 

17 . 83 

TIS 

RATE 

TIP 

NEC 

FLEET 

INSTRUCTOR 


RATE 

TIP 

NEC 

FLEET 

INSTRUCTOR 

ETl 

13 

60 

36 

15 

ETl 

30 

48 

48 

1 

ETl 

79 

72 

36 

24 

ET2 

33 

57 

46 

2 

ETl 

8 

42 

24 

9 

ETl 

86 

76 

36 

41 

ET2 

20 

30 

30 

1 

ETl 

48 

72 

48 

36 

ETl 

100 

58 

24 

23 

ETl 

30 

48 

42 

2 

ETl 

33 

53 

18 

24 

Time  in  service  (years) 

Rate/Rating 

Time  in  paygrade  (months) 

Time  in  NEC  ET-1453  (months) 

Time  working  with  NAVMACS  System  in  Fleet  (months) 
Time  as  NAVMACS  Instructor  (months) 
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APPENDIX  B 


HYPOTHESES  TESTING  DATA 

Page 

HYPOTHESES  1 .  B-1 

HYPOTHESIS  3 .  B-4 

HYPOTHESIS  4 .  B-7 

HYPO!  HESIS  5 .  B-12 

HYPOTHESIS  6 .  B-21 

HYPOTHESIS  7 .  B-28 

HYPTOHESIS8-11 .  B-31 

HYPOTHESIS  12 .  B.37 

HYPOTHESIS  13 .  B-40 

HYPOTHESIS  14 . - .  B.43 

HYPOTHESIS  15 .  8-54 

HYPOTHESIS  17 .  B-57 

HYPOTHESIS  18 .  B-60 

HYPOTHESIS  19 .  B-63 

HYPOTHESIS  20 .  B-66 

HYPOTHESIS  21 .  B-74 

HYPOTHESIS  22 .  B-82 

Note.  Hypotheses  2  and  16  were  dropped. 


B-0 


HYP  1  VAR  -  TAE  FINAL  SCORE  FOR  EACH  CASE  -  STUDENTS  &  INSTRUCT 


HEADER  DATA  FCR : 
NUMBER  OF  CASES: 


C:HyP1  LABEL:  TAE  FINAL  SCORE  MEANS 
Z9  NUMBER  OF  VARIABLES:  1 


FOR  EACH  CASE  -S/I 


1 

TAESCORE 

69.68 

6ii<  .88 

3 

76.64 

4 

7(i».83 

5 

69,46 

6 

81  .47 

7 

70.49 

e 

55.88 

R 

75.45 

Jf' 

70.15 

1  1 

66.05 

12 

73.57 

13 

58.61 

14 

56.83 

15 

70.87 

16 

73.49 

17 

75.85 

le 

75.54 

19 

65.63 

Cu 

69.18 

21 

75.34 

22 

61  .81 

23 

66.73 

24 

72.31 

25 

67.35 

'26 

60.78 

27 

76.54 

28 

67.94 

29 

66.67 

S'? 

78.08 

31 

68.05 

32 

71  .01 

33 

68,19 

34 

66  .80 

35 

69.62 

36 

65.84 

37 

72.05 

38 

75.89 

39 

74,63 

Ai) 

76.99 

41 

74  .49 

42 

71  .82 

43 

78.35 

44 

68.74 

45 

76,29 

46 

70,05 

47 

79.77 

48 

70.35 

49 

79.74 

se 

67.65 

51 

66.20 

52 

74.15 

52 

73.51 

54 

60.18 

m  w 

68.85 

56 

61  .39 

57 

62.35 

58 

81  .42 

59 

72 .2i<> 

B-2 


ANALYSIS  OF  VARIANCE 


HEADER  DATA  FOR:  CrTAESCORE  LABEL:  TAE  FINAL  SCORE  MEANS  FOR  EACH  CAGE 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  1 


ONE-WAY  ANOVA 

HYPOTHESIS  1  TtSi  -  ANuVA :  TAE  FINAL  SCORES  -  STUDENT  vs  INSTRUC 


GROUP 

MEAN 

N 

1 

70.396 

48 

2 

73.422 

11 

GRAND  MEAN 

70.960 

59 

VARIABLF  1:  TAESCORE 


SOURCE 

SUM  OF  SQUARES 

D.F  . 

MEAN  SQUARE 

F  RATIO 

PROB . 

BETWEEN 

81 .973 

1 

81 .973 

2.271 

.1373 

WITHIN 

2057 . 1 24 

57 

36.090 

TOTAL 

2139.096 

58 

HYPOTHESIS  3 


N<uc  2  uas  dropped. 


B-4 


HYPOTHESIS  3  VA 

R  -  TIS  AND  AVG  FINAL  SCORE  - 

STUDENTS  6  INSTRUCT 

HEADER 

DATA  FOR 

:  C:HYF3  LABEL:  HYP  3  VAR  - 

AVG  FIN  SCORE  l<  TIS  -  S  i  I 

NUMBER 

OF  CASES 

:  59  NUMBER  OF  VARIABLES;  2 

TIS 

TAESCORE 

1 

1  .50 

69.68 

2 

1  .58 

60.88 

is 

9.25 

76 .64 

_ 

4 

1  .83 

70.83 

c 

2 .00 

69.46 

t> 

2  .0i;‘ 

81  .47 

7 

4.83 

70.49 

E 

1  .63 

55.88 

9 

1  .72 

75.45 

1  W 

1  .85 

70.15 

1  1 

1.41 

66  .05 

12 

1  .63 

73.57 

13 

1.61 

58.61 

1  A 

1  .BA 

56.83 

15 

1  .48 

70.87 

16 

1  .47 

73.49 

17 

1  .60 

75.85 

le 

6.49 

75.54 

19 

1  .52 

65.63 

20 

1  .67 

69.18 

21 

1  .57 

75.34 

nn 

1  .62 

61  .81 

23 

5  .06 

66.73 

24 

1  .53 

72.31 

25 

1  .47 

67.35 

'  26 

7.93 

60.78 

27 

1  .56 

76  .54 

2B 

1  .47 

67.94 

29 

2.14 

66.67 

30 

1  .39 

78.08 

31 

1  .55 

68.05 

32 

1  .39 

71  .01 

33 

1  .30 

68.19 

34 

1  .29 

66.80 

35 

1  .48 

69.62 

36 

1  .54 

65.84 

37 

1  .37 

72.05 

38 

1  .41 

75.89 

39 

1  .40 

74.63 

40 

1  .35 

76.99 

41 

2.50 

74.49 

42 

2.78 

71  .82 

43 

1  .78 

78.35 

4A 

2.29 

68.74 

45 

1  .68 

76.29 

- 

46 

3.79 

70.05 

47 

1  .75 

79.77 

4E 

1  .79 

70  .35 

49 

9.00 

79.74 

50 

12.75 

67.65 

51 

15.00 

66.20 

8.75 

74.15 

53 

6  .58 

73.51 

54 

13.83 

80.18 

1 

Be- 

7.42 

68.85 

56 

8.92 

81  .39 

57 

7.50 

62.35 

58 

6.92 

81  .42 

59 

17.83 

72.20 

B-5 

CORRELATION  MATRIX 


HEADER  DATA  FOR:  C;HYP3  LABEL:  HYP  3  VAR  -  AVB  FIN  SCORE  tc  TIS  -  S  «<  I 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES:  2 


TESTING  OF  HYPOTHESIS  3  -  CORRELATION  OF  TIS  AND  FINAL  SCORE 


TIS  F INSCORE 
TIS  1.00000 

FINSCORE  .13676  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .21638 
CRITICAL  VALUE  (2-tail ,  .05)  =  +/-  .25614 

N  =  59 


B-6 


HYPOTHESIS  4 


B-7 


HYPOTHESIS  4  VAR  -  SCHOOL  FINAL  SCORE  vs  TAE  FINAL  SCORE  -  STDNT 


HEADER  DATA  FOR;  C;HYP4  LABEL:  HYP  4  VAR  -  TAE  FIN  SC  vs  SCHOOL  FIN  SC 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


SCLSCORE 

TAESCORE 

1 

92.35 

69.68 

2 

90 . 66 

60.88 

3 

96.07 

76.64 

4 

96  .01 

70.83 

cr 

92  .62 

69.46 

6 

95.85 

81  .47 

7 

9=^ .  1 5 

70.49 

8 

95.43 

55.88 

9 

96.85 

75.45 

l«?i 

98 . 35 

70.15 

1  1 

95  .08 

66  .05 

12 

97.67 

73.57 

13 

94  .46 

58  .61 

14 

96.53 

56  .83 

1? 

93.20 

70.87 

16 

96  .80 

73  .49 

17 

92.22 

75.85 

18 

94 .16 

75.54 

19 

93.06 

65.63 

20 

94  .22 

69.18 

21 

93.80 

75.34 

22 

93.24 

61  .81 

23 

94.98 

66.73 

24 

96.01 

72.31 

25 

93.51 

67.35 

26 

88.73 

60.78 

27 

94  .29 

76.54 

28 

96.79 

67.94 

29 

94.62 

66.67 

30 

98  .07 

78.08 

31 

96.96 

68.05 

32 

94  .77 

71  .01 

33 

94.04 

68.19 

34 

96.93 

66.80 

35 

92.98 

69.62 

36 

98.23 

65.84 

37 

97.93 

72.05 

38 

93.43 

75.89 

39 

96.77 

74.63 

40 

93.09 

76.99 

41 

96.83 

74.49 

42 

92.26 

71  .82 

43 

96  .50 

78.35 

44 

96.12 

68.74 

45 

96.56 

76.29 

46 

95.75 

70.05 

47 

97  .99 

79,77 

48 

95.39 

70.35 

B-8 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP4  LABEL:  HYP  4  VAR  -  TAE  FIN  SC  vs  SCHOOL  FIN  SC 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  4  TESTING  -  CORRELATION  OF  SCHL  FIN  SC  vs  TAE  FIN  SC 


SCLSCORE  TAESCORE 
SCLSCORE  1  .00000 
TAESCORE  .30181  1.00000 


CRITICAL  VALUE  (1-TAlL,  .05)  =  +  Or  -  .24045 
CRITICAL  VALUE  (2-tai1 ,  .05)  =  +/-  .28419 

N  =  48 


B-9 


HYP  A  COMP  VAR  -  SCHOOL  COMP  SCORE  vs  STUDENT  TAE  FINAL  SCORE 


HEADER  DATA  FOR:  CrHYPACOMP  LABEL:  HYP  4  COMP  VAR  -  TAE  FIN  SC  vs  SCH  COh 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


COMPSCOR 

TAESCORE 

1 

78.00 

69.68 

n 

.00 

60.88 

3 

9E  .00 

76.64 

A 

8B  .00 

70.83 

82 . 00 

69.46 

6 

90  .00 

81  .47 

7 

90 . 0(?i 

70.49 

6 

88  .0<') 

55.88 

9 

9t)  .0(>i 

75.45 

10 

94  .00 

70.15 

1 1 

78 .00 

66.05 

12 

96 . 00 

73.57 

13 

9(') .  0(') 

58.61 

14 

92 . 00 

56.83 

15 

84  .00 

70.87 

16 

86 .00 

73.49 

17 

88 . 0(;) 

75.85 

18 

84  .00 

75.54 

19 

90 . 00 

65.63 

20 

88.00 

69.18 

21 

74  .00 

75  .34 

22 

80  .  00 

61  .81 

23 

94  .00 

66.73 

24 

88.00 

72.31 

25 

82.00 

67.35 

26 

80 . 00 

60.78 

27 

86  .00 

76.54 

28 

90 . 00 

67.94 

29 

84.00 

66.67 

30 

88.00 

78.08 

31 

86  .00 

68.05 

32 

88.00 

71  .01 

33 

86.00 

68.19 

34 

84  .00 

66.80 

35 

88.00 

69.62 

36 

88.00 

65.84 

37 

90.00 

72.05 

38 

84  .00 

75.89 

39 

90.00 

74.63 

40 

92.00 

76.99 

41 

94  .00 

74.49 

42 

86  .00 

71  .82 

43 

90.00 

78.35 

44 

82.00 

68.74 

45 

84 .0(ii 

76.29 

46 

86 .0^1 

70.05 

47 

96.00 

79.77 

48 

84  . 00 

70.35 

B-lO 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR;  C:HYP4C0MP  LABEL:  HYP  4  COMP  VAR  -  TAE  FIN  SC  vs  SCH  COM 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


HYP  4  COMP  TESTING  -  CORRELATION  OF  SCHL  COMP  SC  vs  TAE  FINAL  SC 


COMPSCOR  TAESCORE 
COMPSCOR  1 .©0000 
TAESCORE  .J7311  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .24045 
CRITICAL  VALUE  (2-tail  ,  .05)  =  +/-  .28419 

N  =  48 


B-11 


HYPOTHESIS  5 


B^12 


HYPOTHESIS  5  VARIABLES  -  SUBSYSTEM  1  -  SCHOOL  FINAL  1  vs  TAE  AVG 


HEADER  DATA  FOR:  C:HYP5SS1  LABEL:  HYP  5  -  SSI  VAR  -  TAE  AVG  SC  vs  FINAL  3 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


SCHL-SSl 

TAE-SSl 

1 

72 .00 

70.39 

70  .00 

80  .00 

3 

9c. .  00 

75.21 

4 

74  .00 

88.50 

C 

7^  .00 

61  .04 

6 

94 .00 

87.55 

7B  .00 

80  .38 

B 

86 .00 

44 . 95 

9 

9<'l  .  ('K'l 

86  .05 

98  .Oi;> 

78 .13 

1  1 

94 . 00 

83.35 

13 

9l'l  .  ('li;' 

88  . 0 1 

13 

84  .00 

74.36 

14 

83 

41  .51 

15 

76 . 0(;1 

78.51 

16 

90  .00) 

89  .95 

17 

78 .00 

78  .06 

18 

78.00 

89  .75 

19 

86  .00 

68.61 

20 

88.00 

82.95 

21 

82.00 

89.95 

22 

90 . 00 

46.41 

23 

94  .00 

88.16 

24 

100.00 

91  .65 

25 

92.00 

87.20 

26 

74  .00 

69.90 

27 

86  . 00 

80.05 

28 

96 .00 

85.65 

29 

90  .00 

84.85 

30 

96.00 

74.85 

31 

96.00 

82.26 

32 

88.00 

85.38 

33 

92.00 

78.59 

34 

86  .00 

73.79 

35 

88.00 

58.65 

36 

94.00 

78.89 

37 

98.00 

84.70 

38 

96.00 

79.04 

39 

92.00 

83.95 

4e 

78.00 

82.70 

41 

98.00 

88.80 

42 

86.00 

70.50 

43 

96.00 

87.55 

44 

94  .00 

81  .44 

45 

96.00 

82.44 

46 

80  .00 

73.61 

47 

94  .00 

90.35 

48 

84  .00 

74.15 

B'13 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  CrHYPSSSl  LABEL:  HYP  5  -  SSI 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


VAR  -  TAE  AVG  SC  vs  FINAL 


^  TEST  —  SSI..  CORRELATION  SCHOOL  FINAL  1  vs  TAE  AVERAGE  SC 


SCHL-SSl  TAE-SSl 
SCHl-ESI  1 .00000 
TAE-SSl  .27704  1 .00000 


CRITICAL  VALUE  (1-TAIL, 
CRITICAL  VALUE  <2-tal1, 

N  =  48 


.05)  =  +  Or  -  .24045 

.05)  =  +/-  .28419 


B-14 


HYPOTHESIS  5  VARIABLES  -  SUBSYSTEM  2  -  SCHOOL  FINAL  2  vb  TAE  AVG 


HEADER  DATA  FOR:  CiHYP5SS2  LABEL*  HYP  5  -  SS2  VAR  -  TAE  AVG  SC  vs  FINAL  I 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


SCHL-SS2 

TAE-SS2 

5 

i 

82  .00 

64 . 70 

90 . 00 

52.69 

3 

9B.00 

82.21 

4 

92 

77.27 

C 

94 . 00 

75.15 

L 

96.00 

77 . 59 

7 

10C'l  .00 

69.61 

8 

84 . 00 

59.34 

9 

94  .HU;l 

72  .44 

1<?> 

9(?)  .  00 

71  .37 

1  1 

84  . 00 

70.06 

12 

86  .  i-h;) 

77.43 

13 

88 . 0<') 

56.20 

14 

92  .00 

54.27 

15 

88 

67.65 

16 

92 

84.05 

17 

92 .00 

72.12 

18 

86 .00 

74.93 

19 

86  .00 

67.75 

2€) 

92.00 

68.97 

21 

90  .00 

75.37 

22 

88.00 

69.03 

23 

94  .00 

67.89 

24 

92.00 

69.44 

25 

96  .00 

62.64 

26 

84  .00 

63.37 

27 

84  .00 

79.93 

28 

96  .00 

70.55 

29 

90  .00 

61  .45 

30 

94.00 

83.69 

31 

90.00 

62.75 

32 

84  .00 

67.82 

33 

94  .00 

64.56 

34 

94  .00 

66.54 

35 

92.00 

79.11 

36 

98.00 

65.87 

37 

94.00 

73.25 

38 

92.00 

82.58 

39 

84.00 

73.94 

40 

88.00 

79.85 

41 

96.00 

74.78 

42 

96.00 

75.45 

43 

92.00 

80.83 

44 

92.00 

75.13 

45 

98.00 

74.45 

46 

90  .00 

73.15 

47 

88.00 

81  .36 

48 

86  .00 

69.01 

B-15 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP5SS2  LABEL:  HYP  5  -  SS2  VAR  -  FINAL  2  vb  TAE  AVG  SC 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  2 


HYP  5  TEST  -  SS2  -  CORRELATION  SCHOOL  FINAL  2  vb  TAE  AVERAGE  SC 


SCHL-SS2  TAE-SS2 
SCHL-SS2  1 .00000 
TAE-SS2  .17579  1.00000 


critical  VALUE  (1-TAIL,  .05)  «=  +  Or  -  .24045 
CRITICAL  VALUE  (2-tail ,  .05)  =  +/-  .28419 

N  =  48 


B-16 


HYPOTHESIS  5  VARIABLES  -  SUBSYSTEM  3  -  SCHOOL  FINAL  3  ^'s  TAE  AVG 


HEADER  data  FOR;  C;HYP5SS3  LABEL:  HYP  5  -  SS3  VAR  -  FINAL  3  vs  TAE  AVG  SC 
NUMBER  OF  CASES;  48  NUMBER  OF  VARIABLES:  2 


SCHL-SS3 

TAE-SS3 

1 

86.70 

98.35 

8£  .70 

93.55 

3 

93.30 

98.65 

4 

86.70 

98.25 

93.30 

93  .00 

6 

1 0(i)  . 

91  .90 

7 

93.30 

95.50 

8 

86.70 

93.60 

9 

1 0(;!  .0';i 

99.75 

1(?> 

93.30 

99  .00 

1 1 

93.30 

98.75 

12 

93.30 

98.95 

13 

100 .00 

93.40 

14 

1  01?)  .  0(?) 

96.75 

iZ 

86.70 

98  .90 

16 

1  00  .  <?)^) 

73.35 

17 

86  .70 

85  .90 

18 

93.30 

99.15 

19 

93.30 

95  .95 

2<i) 

100.00 

97.00 

21 

93.30 

91  .95 

22 

93.30 

74.86 

23 

93.30 

91  .45 

24 

100 .00 

88.25 

25 

73.30 

92.55 

26 

100 .00 

93.45 

27 

93.30 

95.70 

28 

100 .00 

80.25 

29 

93.30 

86  .80 

30 

93.30 

96.75 

31 

86  .70 

98.40 

32 

86.70 

83.95 

33 

86.70 

86.20 

34 

93.30 

99.00 

35 

100.00 

84.50 

36 

93.30 

80.86 

37 

93.30 

99.50 

38 

86.70 

87.20 

39 

B3.30 

99.25 

40 

86  .70 

95.50 

41 

93.30 

97.65 

42 

73.30 

95.50 

43 

93.30 

96.35 

44 

100 .00 

79.31 

45 

86.70 

99.45 

46 

100.00 

97.15 

47 

100 .00 

98.75 

48 

86 .70 

97.05 

B^17 


CORRELATION  MATRIX 


HEADER  DATA  FOR: 
NUMBER  OF  CASES; 


HYP  5  TEST  -  SS3 


SCHL-SS3 
SCHL-SS3  1 .00000 
TAE-SS3  -.18146 


CRITICAL  VALUE  (1 
CRITICAL  VALUE  (2 

N  =  48 


C:HYP5SS3  LABEL:  HYP  5  -  SB3  VAR  -  FINAL  3  vs  TAE  AVG  SC 
48  NUMBER  OF  VARIABLES:  2 


-  CORRELATION  SCHOOL  FINAL  3  vs  TAE  AVERAGE  SC 


TAE-SS3 

1 .00000 


-TAIL,  .05)  =  +  Ot - .24045 

-tail ,  .05)  =  +/-  .28419 


B-18 


HYPOTHESIS  5  VARIABLES  -  SUBSYSTEM  4  -  SCHOOL  FINAL  5  vs  TAE  AVG 

HEADER  DATA  FOR:  C:HYP5SS4  LABEL:  HYP  5  -  SS4  VAR  -  FINAL  5  vs  TAE  AVG  SC 
NUMBER  OF  CASES:  4B  NUMBER  OF  VARIABLES;  2 


SCHL-SS4 

TAE-SS4 

1 

75  .00 

60.24 

o 

.00 

41  .85 

3 

10C  .00 

33.75 

4 

1  0(1  .  00 

.00 

«- 

95  .00 

31  .54 

6 

100 .00 

80  ,50 

7 

1 00 .00 

39.14 

8 

95 .00 

15.25 

9 

1 00 . 00 

52.58 

li'i 

100 .00 

28.39 

1  1 

1  00  .  <'>0 

.<•’0 

12 

1  0I'>  . 

18.29 

13 

95  .  t'Hil 

17.69 

14 

95 .00 

42,45 

12. 

80 .00 

48.09 

16 

95 . 00 

14.94 

17 

9(1 .0(1 

78.48 

18 

75 . 00 

40.15 

19 

100 .00 

23.85 

20 

80.00 

28.44 

21 

100 .00 

44  .00 

21 

100 .00 

35.25 

23 

95.00 

15.95 

24 

95.00 

48.48 

25 

100.00 

41 .18 

26 

90  .00 

8.59 

27 

95.00 

40.29 

28 

90 ,00 

27.48 

29 

90  .00 

49.19 

30 

95.00 

40.24 

31 

90  .00 

44.68 

32 

90.00 

56.44 

33 

95.00 

54.29 

34 

95.00 

28.67 

35 

90  .00 

27.74 

36 

95.00 

37.64 

37 

95.00 

27.14 

36 

95.00 

34.67 

39 

95.00 

43.44 

40 

B5.00 

41  .29 

41 

95.00 

35.88 

42 

80.00 

34.94 

43 

95.00 

41  .23 

44 

90,00 

19.90 

45 

90  .00 

54.39 

46 

100.00 

26.98 

47 

05 .00 

43.89 

48 

1 00 . 00 

45.24 

B-19 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP5SS4  LABEL;  HYP  5  -  SS4  VAR  -  FINAL  5  vs  TAE  AVG  SC 
NUMBER  OF  CASES;  4B  NUMBER  OF  VARIABLES:  2 


HYP  5  TEST  -  F?;  -  CORRELATION  SCHOOL  FINAL  5  vs  TAE  AVERAGE  SC 


SCHL-SS4  TAE-SS4 
SCHL-SS4  1 .00000 
TAE-SS4  -.21972  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .24045 
critical.  VA:_Ut  <2-tail  ,  .05)  =  .28419 

N  =  48 


B-20 


HYPOTHESIS  6 


B-21 


hypothesis  6  VAR  -  TAE  AVG  SC/AFOT/BS/AR/MK/E 1 /ASVAB-i /ASVAS-T 


HEADER  DATA  FOR:  C;HYP&VAR  LABEL:  HYPOTHESIS  6  VARIABLES 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  8 


STUDENT  t,  INSTR 


TAESCORE 

AFDT 

GS 

AR 

MK 

El 

ASVAB-1 

I 

69.68 

74.00 

62.00 

58.00 

62.00 

44.00 

168.00 

2 

60.88 

99  .00 

64  .00 

66.00 

68.00 

63.00 

195.00 

3 

76 . 64 

83.00 

58 . 00 

64.00 

64.00 

62  .00 

184.00 

4 

70.83 

91  .00 

62.00 

62.00 

65.00 

53.00 

180.00 

C 

69.46 

72  .00 

60.00 

55.00 

52.00 

55.00 

167.00 

6 

81  .47 

83  .00 

62.00 

62.00 

66  .00 

64  .00 

192.00 

7 

70.49 

98.00 

65.00 

67.00 

71  .00 

56.00 

1 92 . 00 

8 

55.88 

86  .00 

60 .00 

59.00 

65.00 

68.00 

193  .00 

9 

75.45 

96  .00 

63  .00 

63 .00 

66  . 00 

67  .00 

196  .00 

1  i;i 

70 . 1 5 

85 . 00 

62.00 

62.00 

62  .00 

53 . 00 

177.00 

1  1 

66.05 

77 .00 

55.00 

56.00 

55 . 00 

62  , 00 

172.00 

12 

73 . 57 

72 . 00 

56 . 00 

58.00 

63 . 00 

67,00 

186 .00 

13 

58.61 

64  .  «i0 

51  .00 

58  .00 

66.00 

55.00 

172.00 

14 

56.83 

70 . 0I'> 

58.00 

58.00 

61.00 

55  .00 

174 .00 

1? 

70.87 

81  .00 

58.00 

62.00 

64  .00 

62.00 

184.00 

16 

73.49 

99 . 00 

63.00 

64.00 

64  .00 

69.00 

196,00 

17 

75.85 

96  .00 

62.00 

60.00 

58.00 

67.00 

187.00 

18 

75  .54 

78.00 

52.00 

60.00 

61  .00 

54.00 

167  .00 

19 

65.63 

64  . 00 

53.00 

59.00 

63  .00 

53 .00 

169.00 

2e) 

69.18 

72  .00 

62.00 

55.00 

58.00 

64.00 

164.00 

21 

75.34 

62.00 

60.00 

50.00 

57.00 

58  .00 

175.00 

61  .81 

54  .00 

49.00 

62.00 

66.00 

67.00 

182.00 

23 

66.73 

74  .00 

48.00 

66.00 

68.00 

61  .00 

177.00 

24 

72.31 

76.00 

62.00 

63.00 

57.00 

67.00 

186.00 

25 

67.35 

73.00 

56.00 

59.00 

64.00 

64  .00 

164 .00 

26 

60 . 78 

68  .00 

62.00 

53.00 

52.00 

65.00 

179.00 

27 

76.54 

87.00 

58 .00 

58.00 

64  .00 

64  .00 

166.00 

28 

67.94 

86 . 00 

65.00 

58.00 

64.00 

64  .00 

193.00 

29 

66.67 

91  .00 

62.00 

62.00 

60 . 00 

58.00 

180 .00 

3^ 

78.08 

87.00 

67.00 

62.00 

68.00 

68  .00 

203.00 

31 

68.05 

75.00 

62.00 

64.00 

64  .00 

56.00 

182.00 

32 

71  .01 

94  .00 

67.00 

66.00 

/>3.00 

66.00 

196.00 

33 

68.19 

93.00 

58.00 

63.00 

61  .00 

62.00 

181 .00 

34 

66  .80 

93.00 

63.00 

64.00 

64  .00 

64.00 

191  .00 

35 

69 .62 

82  .00 

65.00 

58.00 

58.00 

62.00 

185.00 

36 

65.84 

80.00 

63.00 

62.00 

55.00 

64.00 

182.00 

37 

72.05 

98.00 

65.00 

63.00 

60.00 

69.00 

194.00 

38 

75.89 

75.00 

65.00 

62.00 

61  .00 

67.00 

193.00 

39 

74.63 

51  .00 

44.00 

55.00 

55.00 

51  .00 

150.00 

40 

76.99 

82.00 

62.00 

60.00 

61  .00 

62.00 

165.00 

41 

74  .49 

87.00 

64.00 

66.00 

68.00 

54.00 

186.00 

42 

228.00 

75.00 

62.00 

50.00 

58.00 

58.00 

178.00 

43 

78 . 35 

84.00 

58.00 

60.00 

64.00 

67.00 

189 .00 

44 

68 .74 

96  .00 

58.00 

64.00 

61  .00 

64.00 

183.00 

45 

76.29 

88.00 

64.00 

61  .00 

65.00 

51  .00 

180 .00 

46 

70.05 

73.00 

53.00 

56 .00 

52.00 

67.00 

172.00 

47 

79.77 

93,00 

60.00 

62.00 

66.00 

55.00 

181 .00 

48 

70.35 

40  ,00 

53.00 

51  .00 

60.00 

67.00 

180.00 

B-22 


ASVAB-T 

1 

226.00 

n 

261 .00 

3 

248.00 

4 

242.00 

c; 

222.00 

6 

254.00 

7 

259.00 

6 

252.00 

9 

259.00 

K- 

239.00 

1  ] 

228.00 

1  3 

244.00 

13 

230.00 

14 

232.00 

i? 

246 .00 

16 

260.00 

17 

247.00 

18 

227 .00 

19 

228  .00 

2k) 

239.00 

21 

225.00 

0-7 

244 .00 

23 

243.00 

24 

249.00 

oc 

243.00 

26 

232.00 

27 

244 .00 

28 

251 .00 

29 

242.00 

30 

265.00 

31 

246.00 

32 

262.00 

33 

244.00 

34 

255.00 

35 

243.00 

36 

244 .00 

37 

257.00 

38 

255.00 

39 

205.00 

40 

245.00 

41 

252,00 

42 

228.00 

43 

249.00 

44 

247.00 

45 

241 .00 

46 

228,00 

47 

243.00 

48 

231 .00 

B-23 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  CtHYPAVAR  LABEL;  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  B 


HYPOTHESIS  6  TEST  -  CORRELATION  BETWEEN  TAE  FINAL  SCORE  AND  AFQT 


TAESCORE  AFQT 
TAESCORE  1 .00000 
AFQT  -.00398  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Oi - .24045 

CRITICAL  VALUE  (2-taa1 ,  ,05)  =  +/-  .28419 

N  =  48 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP6VAR  LABEL;  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  8 


HYPOTHESIS  6  TEST  -  CORRELATION  BETWEEN  TAE  FINAL  SCORE  AND  GS 


TAESCORE  GS 

TAESCORE  1 .00000 
GS  .11462  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .24045 
CRITICAL  VALUE  .05)  *  ♦/-  .28419 

N  -  48 


B-24 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP6VAR  LABEL:  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  8 


HYPOTHESIS  6  TEST  -  CORRELATION  BETWEEN  TAE  FINAL  SCORE  AND  AR 


TAESCORE  AR 

TAESCORE  1  .00001/) 

AR  -.32510  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or - .24045 

CRITICA;_  VALUE  (2-tail  ,  .05)  =  +/-  .28419 

N  =  48 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP6VAR  LABEL:  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  8 


HYPOTHESIS  6  TEST  -  CORRELATION  BETWEEN  TAE  FINAL  SCORE  AND  MK 


TAESCORE  MK 

TAESCORE  1 .00000 
MK  -.10088  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .24045 
critical  value  (2-tai1  ,  .05)  =  +/-  .28419 

N  =  48 


B-25 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP6VAR  LABEL:  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  B 


HYPOTHESIS  6  TEST  -  CORRELATION  BETWEEN  TAE  FINAL  SCORE  AND  El 


TAESCORE  El 

TAESCORE  3  .^>0000 
El  -.06673  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .24045 
CRITICAL  VALUE  (2-tail  ,  .05)  =  +/-  .28419 

N  =  48 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP6VAR  LABEL:  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
NUMBER  OF  CASES:  48  NUMBER  OF  VARIABLES:  8 


HYPOTHESIS  6  TEST  -  CORRELATION  BETWEEN  TAE  FINAL  SCORE  ^  ASVABl 


TAESCORE  ASVAB-1 
TAESCORE  1 .00000 
ASVAB-1  -.02672  1.00000 


CRITICAL  VALUE  (1-TAlL,  .05)  =  +  Oi - .24045 

CRITICAL  VALUE  (2-tai1  ,  .05)  =  ♦/-  .28419 

N  =  48 


B-26 


CORRELATION  MATRIX 


HEADER  DATA  FOR: 
NUMBER  OF  CASES: 


HYPOTHESIS  6  TEST 


TAFSCORE 
TAE  SCORE  1 
ASVAB-T  -.13055 


critical  value  (1 

CRITICAL  VALUE  (2 
N  =  48 


C:HYP6VAR  LABEL;  HYPOTHESIS  6  VARIABLES  -  STUDENT  ONLY 
48  NUMBER  OF  VARIABLES:  B 


-  CORRELATION  BETWEEN  TAE  FINAL  SCORE  ?<  ASVABT 

ASVAB-T 

1 .00000 


TAIL,  .05)  =  +  Or  -  .24045 
tail,  .05)  =  +/-  .28419 


B-27 


HYPOTHESIS  7 


B-28 


HYPOTHESIS  7  VAR  -  TAE  AVERAGE  SCORE  vs  NUMBER  VALID  CHECKS 


S/1 


HEADER  DATA  FDR: 
NUMBER  OF  CASES; 


C:HYF7  LABEL:  HYP  7  VAR  - 
59  NUMBER  OF  VARIABLES:  2 


TAE  AVG  SC  vs  INVALID  CHECKS 


TAESCORE 

INVALCKS 

1 

69.68 

.00 

60.88 

.00 

3 

76.64 

.00 

4 

70.83 

.29 

5 

69.46 

.21 

b 

81  .47 

.00 

7 

70.49 

.21 

8 

55.88 

.00 

9 

75  .45 

.00 

Idi 

70.15 

.14 

1  1 

66.05 

.00 

12 

73.57 

.00 

1  3 

58.61 

.00 

14 

56.83 

.29 

15 

70.87 

.00 

16 

73.49 

.21 

17 

75.85 

.00 

18 

75.54 

.00 

19 

65.63 

.00 

2i> 

69.18 

.07 

21 

75.34 

.14 

22 

61  .81 

.00 

23 

66  .73 

.00 

24 

72.31 

.00 

25 

67.35 

.00 

26 

60.78 

.00 

27 

76.54 

.00 

28 

67.94 

.07 

29 

66.67 

.00 

3«i 

78.08 

.00 

31 

68.05 

.36 

32 

71  .01 

.00 

33 

68.19 

.14 

34 

66  .80 

.14 

35 

69.62 

.07 

36 

65  .84 

.00 

37 

72.05 

.21 

38 

75.89 

.00 

39 

74  .63 

.00 

40 

76.99 

.00 

41 

74.49 

.07 

42 

71  .82 

.21 

43 

78.35 

.00 

44 

68.74 

.00 

45 

76.29 

.14 

46 

70.05 

.07 

47 

79.77 

.00 

48 

70.35 

.00 

49 

79.74 

.00 

50 

67.65 

.00 

51 

66.20 

.57 

52 

74.15 

.00 

53 

73.51 

.00 

54 

80.18 

.07 

55 

68.85 

.14 

56 

81  .39 

.00 

57 

62.35 

.00 

58 

81  .42 

.00 

59 

72.20 

.07 

B'29 


CORRELATION  MATRIX 


HEADER  DATA  FOR;  C;HYP7  LABEL;  HYP  7  VAR  -  TAE  AVG  SC  vs  INVALID  CHECKS 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  7  TESTING ;' CORRELATION  OF  TAE  FIN  SC  &  #  INVALID  CHKS 


TAESCORE  INVALCKS 
TAESCORE  ! 

INVALCKS  -.17107  1.00000 


CRITICAL  VALUE  (1-TAlL,  .05)  =  +  Or  -  .21638 
CRITICAL  VALUE  (2-tail  ,  .05)  =  +/-  .25614 


HYPOTHESES  8 


LIST  OF  VAFIA&LES  TO  TEST  HYPOTHESES  8-11  -  STUDENTS  It  INSTRUCT 


HEADER 

DATA  FOR 

CD 

Li. 

>- 

X 

u 

LABEL 

:  VAR  FDR 

HYPOTHESES 

NUMBER 

OF  CASES 

:  59  NUMBER 

or  VARIABLES:  5 

7  AESCDR'E 

ILL  AF'PR  INC  SQLU 

REDUN  CK 

PROOF  PT 

1 

69.68 

.21 

2.71 

.29 

.64 

60.88 

.07 

5  .21 

.07 

.43 

3 

76.64 

.14 

3.07 

.14 

.64 

4 

70.83 

.07 

4.71 

.07 

.93 

69.46 

.14 

2.43 

.14 

.79 

t 

81  .47 

.07 

.64 

.00 

.93 

7 

70.49 

.36 

2.14 

.00 

.43 

8 

55. 8B 

.29 

10.64 

.<•7 

.21 

R 

75.4  5 

.<'10 

1  .64 

.00 

.36 

!>■ 

70.15 

.  1  4 

3.43 

.36 

.50 

1  1 

66.05 

.71 

4.57 

.07 

.43 

1  2 

73.57 

.  1  4 

3.00 

.00 

.50 

13 

58.61 

,14 

5.71 

.57 

.43 

14 

56.83 

.  1  4 

5.71 

.00 

.43 

If. 

70.87 

.  1  4 

2.21 

.50 

.50 

16 

73.49 

.  36 

1  ,07 

.29 

1  .07 

17 

75.85 

.  01.-I 

3.36 

.00 

.50 

18 

75 . 54 

.29 

2.64 

.29 

.57 

19 

65.63 

.07 

6.71 

.00 

,36 

2^1 

<.,9.18 

.29 

3.07 

.29 

.43 

21 

75.34 

.14 

2.57 

.29 

1  .36 

61  .81 

.43 

5,50 

.00 

.36 

23 

66.73 

.36 

8.86 

.00 

.43 

24 

72.31 

•  2 1 

1  .14 

.07 

.43 

Otr 

67.35 

.29 

3.21 

.07 

.50 

26 

60.78 

.43 

8.93 

.00 

.29 

27 

76.54 

.21 

1  .36 

.07 

.57 

28 

67  .94 

.29 

1  .86 

.7: 

.64 

29 

66.67 

.29 

1  .93 

.07 

.43 

78.08 

,00 

2.86 

.07 

1.14 

31 

68.05 

.  00 

5.43 

1.00 

.36 

32 

71  .01 

,43 

4.79 

.14 

.29 

33 

68.19 

.14 

4.29 

.00 

.36 

34 

66  . 80 

.14 

1  .43 

.21 

.71 

35 

69,62 

.  0<'> 

2,07 

.21 

.79 

36 

65.84 

.00 

3.07 

.36 

.57 

37 

72.05 

.14 

2.79 

.50 

.64 

38 

75.89 

.07 

.86 

.36 

1 .14 

39 

74,63 

.00 

1  .29 

.21 

.64 

40 

76.99 

.00 

1  .79 

.07 

.50 

41 

74.49 

.00 

1  .86 

.14 

.57 

42 

71  ,82 

.07 

4.36 

.29 

.43 

43 

78 . 35 

.00 

1  .14 

.07 

.57 

44 

68.74 

.21 

3. •’9 

.43 

.64 

45 

76.29 

.14 

2.43 

.0 

.64 

46 

70  05 

.21 

2.86 

.77 

.64 

47 

->9.77 

.07 

1 .07 

.36 

.64 

40 

70,35 

.36 

1  .36 

.21 

.64 

49 

79 . 74 

.07 

1  .57 

.00 

.93 

50 

67.65 

.21 

4.50 

.07 

.57 

5l 

66  .20 

.00 

4.66 

.21 

.57 

74.15 

.29 

.71 

.29 

.71 

53 

73.51 

.43 

1  .79 

.50 

.93 

54 

80.18 

.14 

4.6* 

.00 

.93 

68.85 

.07 

0.21 

.43 

.50 

56 

81  .39 

.00 

2.00 

.14 

1  .00 

57 

62 . 35 

.07 

3.71 

.14 

.73 

58 

81  ,42 

,00 

.50 

.07 

1  .21 

59 

72 . 20 

00 

4.14 

.07 

.71 
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CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP8-11  LABEL:  VAR  FOR  HYPOTHESES  8-11  -  S 

NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  5 


HYP  e  TEST  -  CORRELATION:  TAE  FINAL  SCORE  «<  ILLOGICAL  APPROACHES 


TAESCORE  ILL  APPR 
TAE  SCOPE  1  .C!>0&00 
III  A^FFn  —  .34(?)57  1 


critical.  VALUE  (l-TAIL,  .(?'5)  =  +  Or  -  .2163B 
CRITICAL  VALUE  (2-tail,  .V5)  =  +/-  .25614 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C;HYPB-n  LABEL:  VAR  FOR  HYPOTHESES  8-11  -  S  «<  I 

NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  5 


HYP  9  TEST  -  CORRELATION:  TAE  FINAL  SCORE  «<  INCORRECT  SOLUTIONS 


TAESCORE  INC  SOLU 
TAE  SCORE  1 

INI  SOLU  -.69676  1.00000 


CRITICAL  VALUE  <1-^AIL,  .05)  =  +  Or  -  .21638 
CRITICAL  VA^UE  (2-tail ,  .05)  =  +/-  .25614 

N  =  59 


B-34 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYR8-11  LABEL:  VAR  FOR  HYPOTHESES  8-11  -  S  «<  I 

NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  5 


HYP  1<(>  TEST  -  CORRELATION:  TAE  FINAL  SCORE  &  REDUNDANT  CHECKS 


lAESCORE  REDUN  CK 
TAE  SCORE  1  .0ei0(>)ro 
REDUN  Cf  -.08543  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Oi - .21638 

CRITICAL  VALUE  (C-tail  ,  .05)  =  +/-  .25614 

N  =  59 


B-35 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP8-11  LABEL:  VAR  FOR  HYPOTHESES  8-11  -  S  I 

NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  5 


HYF  11  TEST  -  CORRELATION:  TAE  FINAL  SCORE  PROOF  POINTS 


TAESCORE  PROOF  PT 
TAE  SCORE  1 

PROOF  PT  .56097  1  .00000 


CRITICAL  VALUE  (l-TAIL,  .05)  =  ■*-  Or  -  .21638 
CRITICAL  VALUE  (2-tail ,  .05)  =  +/-  .25614 

N  =  59 


B-36 


HYPOTHESIS  12 


B-37 


HYPOTHESIS  12  VAR  -  TAE  FINAL  SCORE  v»  AVERAGE  NUMBER  OF  TESTS 


HEADER  DATA  FOR:  CiHYP12  LABEL:  HYP  12  VAR  -  TAE  SC  v»  NUMBER  OF  TESTS 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


TAESCORE 

•  TESTS 

1 

69.68 

5.08 

2 

60.88 

4.97 

3 

76.64 

3.12 

4 

70.83 

3.55 

5 

69.46 

4.86 

6 

81  .47 

2.75 

7 

70.49 

2.86 

8 

55.88 

5.45 

9 

75.45 

2.77 

l<i> 

70.15 

3.62 

1 1 

66.05 

5.01 

12 

73.57 

3.32 

13 

58.61 

5.78 

14 

56.83 

3.82 

15 

70.87 

4.16 

16 

73.49 

3.60 

17 

75.85 

3.40 

18 

75.54 

4.23 

19 

65.63 

3.82 

2C' 

69.18 

3.11 

21 

75.34 

4.85 

22 

61  .81 

3.08 

23 

66.73 

4.87 

24 

72.31 

2.55 

25 

67.35 

3.91 

26 

60.78 

4.55 

27 

76.54 

3.66 

28 

67.94 

3.68 

29 

66.67 

3.32 

3€) 

78.08 

2.92 

31 

68.05 

4.67 

32 

71 .01 

3.39 

33 

68.19 

4.06 

34 

66.80 

3.86 

35 

69.62 

4.53 

36 

65.84 

3.56 

37 

72.05 

4.06 

38 

75.89 

3.20 

39 

74.63 

3.46 

40 

76.99 

2.99 

41 

74.49 

2.90 

42 

71 .82 

3.31 

4? 

78.35 

3.11 

44 

68.74 

5.53 

45 

76.29 

3.03 

46 

70.05 

3.27 

47 

79.77 

3.56 

48 

70.35 

5.38 

49 

79.74 

3.08 

50 

67.65 

4.02 

51 

66.20 

5.08 

52 

74.15 

3.56 

53 

73.51 

4.90 

54 

80.18 

3.65 

55 

68.85 

3.37 

56 

81  .39 

2.68 

57 

62.35 

5.51 

58 

81  .42 

3.99 

59 

72.20 

4.27 
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CORRELATION  MATRIX 


HEADER  DATA  FOR:  CiHYP12  LABEL:  HYP  12  VAR  -  TAE  SC  vs  NUMBER  OF  TESTS 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  12  -  CORRELATION:  TAE  FINAL  SCORE  vs  NUMBER  OF  TESTS 


TAESCORE  #  TESTS 
TAE SCORE  1 .00000 
#  TESTS  -.55201  1  .0000(^ 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .21638 
CRITICAL  VALUE  <2-tail  ,  .05)  =  +/-  .25614 

N  =  59 


B-39 


HYPOTHESIS  13 


B-40 


DIFFICULTY  TOTAL  SCORES  &  2  SCORES  -  EPISODES  IN  ORIGINAL  ORDER 


HEADER  DATA  FOR:  CtZSCORES  LABEL:  DIFF  Z  SCORES  -  EPs  IN  Oi.IGINAL  ORDER 
NUMBER  OF  CASES:  M  NUMBER  OF  VARIABLES:  2 


EPISODE 

Z  SCORES 

1 

2806 .00 

-.81 

2 

2946 .00 

-.77 

3 

2345.00 

-.94 

4 

491“'. 00 

-.19 

5 

8466.00 

.85 

6 

9357.00 

1.11 

7 

1625.00 

-1 .15 

8 

1686.00 

-1 .14 

9 

9636.00 

1  .20 

10 

12159.00 

1  .94 

1 1 

5393.00 

-.05 

12 

3093.00 

-.72 

13 

8049.00 

.73 

14 

531 1 .00 

-.07 

B-41 


HYPOTHESIS  13  VARIABLES  -  TAE  EPISODE  AVERAGE  TIME  vs  Z  SCORE 


HEADER  DATA  FOR:  C;HYP13  LABEL:  HYP  12  VAR  -  EPISODE  AVG  TIME  vs  Z  SCORE 
NUMBER  OF  CASES:  14  NUMBER  OF  VARIA.  LES:  2 


AVGTIME 

ZSCORE 

1 

6.90 

-.81 

2 

7.93 

-.77 

3 

1 1  .20 

-.94 

4 

20.12 

-.19 

tr 

21  .76 

.85 

6 

22.80 

1.11 

7 

3.54 

-1 .15 

8 

3.90 

-1 .14 

9 

40.31 

1  .20 

10 

46.29 

1  .94 

1  1 

17.54 

-.05 

12 

10.37 

-.72 

13 

35.83 

.73 

14 

21  .75 

-.07 

-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP13  LABEL:  HYP  12  VAR  -  EPISODE  AVG  TIME  vs  Z  SCORE 
NUMBER  OF  CASES:  14  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  13  -  CORRELATION;  EPISODE  DIFFICULTY  vs  EPISODE  TIME 


AVGTIME  ZSCORE 
AVGTIME  1.00000 
ZSCORE  .93051  1  .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  «  +  Or  -  .45900 
CRITICAL  value  <2-t«il  ,  .05)  -  ♦/-  .53067 

N  «  14 


B-42 


HYPOTHESIS  14 


B-43 


HYPOTHESIS  14  VAR  -  LEVEL  1  DIEF  -  TAE  FIN  SC  vs  TAE  TOTAL  TIME 


HEADER  DATA  FOR;  CiHYP14-l  LABEL;  HYP  14  -  LEVEL  1  VAR  -  EPISODES  7  AND  8 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES;  2 


TAESCORE 

TAE  TIME 

1 

98 . 35 

1 .50 

o 

93.55 

5.50 

3 

98.65 

1 .50 

4 

98.25 

3.50 

5 

93.00 

5.00 

6 

91  .90 

5.00 

7 

95.50 

3.00 

8 

93.60 

1  .00 

9 

99.75 

.50 

10 

99.00 

2.00 

1  1 

98 . 75 

2.50 

12 

98.95 

1  .50 

13 

93.40 

2.00 

14 

96.75 

1  .50 

15 

98.90 

1  .00 

16 

73.35 

9.50 

1  7 

85.90 

2.00 

le 

99.15 

.50 

19 

95.95 

1  .50 

20 

97.00 

3.00 

21 

91  .95 

2.50 

22 

74.86 

9.50 

23 

91  .45 

3.50 

24 

88.25 

3.50 

25 

92.55 

2.50 

26 

93.45 

4.50 

27 

95.70 

2.00 

28 

80.25 

9.50 

29 

86.80 

9.00 

30 

96.75 

1  .50 

31 

98.40 

2.00 

32 

83.95 

5.50 

33 

86.20 

5.00 

34 

99.00 

2.00 

35 

84.50 

e.oo 

36 

60.86 

12.50 

37 

99.50 

1.00 

38 

87.20 

5.00 

39 

99.25 

1.50 

40 

95.50 

4.00 

41 

97.65 

3.50 

42 

95.50 

3.00 

43 

96.35 

5.50 

44 

79.31 

15.00 

45 

99.45 

.50 

4i 

97.15 

4.50 

47 

98.75 

2.50 

4d 

97.05 

3.50 

49 

98.60 

1  .00 

50 

95.40 

2.00 

51 

97.75 

1 .50 

52 

98.20 

3.00 

53 

97.70 

4.00 

54 

97.90 

3.00 

55 

98.15 

2.50 

56 

84  .75 

5-50 

57 

86.75 

3.50 

58 

97.70 

4.00 

59 

92.35 

3.50 

B-44 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C: HYP 14-1  LABEL:  HYP  14  -  LEVEL  1  VAR  -  EPISODES  7  AND  8 

NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  14  -  LEVEL  1  DIFFICULTY  -  CORR :  FIN  SC  vs  TOTAL  TIME 


TAESCORE  TAE  TIM^ 
TAESCORE  1 .00000 
TAE  TIME  -.81265  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .21638 
CRITICAL  VALUE  (2-tail,  .05)  =  +/-  .25614 

N  =  59 


B-45 


HYPOTHESIS  14  VAR  -  LEVEL  2  DIFF  -  TAE  FIN  SC  v«  TAE  TOTAL  TIME 

HEADER  DATA  FOR;  C:HYF14-2  LABEL;  HYP  14  -  LEVEL  2  VAR  -  EPs  1,  2,  3,  «i  12 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES;  2 


TAESCORE 

TAE  TIME 

1 

75.37 

11.75 

2 

74.64 

12.00 

3 

82.80 

3.25 

4 

91  .12 

9.25 

5 

79.59 

16.00 

6 

78.63 

11  .50 

7 

73.86 

14.25 

8 

52.00 

11.25 

9 

70.45 

6.75 

Id 

78.79 

9.50 

1  1 

81  .98 

4.00 

12 

81  .78 

5.00 

13 

68.40 

7.25 

14 

69.20 

11.75 

15 

79.25 

4.25 

16 

88.30 

12.50 

17 

84.09 

6.50 

18 

78.26 

8.75 

19 

67.21 

15.00 

20 

69.95 

12.75 

21 

90.97 

5.75 

22 

76.07 

13.75 

23 

76.85 

4.75 

24 

73.72 

7.00 

25 

85.69 

5.50 

26 

75.05 

9.25 

27 

89.12 

7.25 

26 

77.63 

9.25 

29 

69.68 

13.00 

30 

84.32 

8.75 

31 

75.67 

5.50 

32 

82.95 

3.25 

33 

79.16 

5.25 

34 

90.57 

9.75 

35 

88.62 

11.75 

36 

77.94 

9.50 

37 

76.13 

9.50 

38 

84.34 

10.00 

39 

63.04 

9.00 

40 

83.89 

7.50 

41 

65.94 

7.50 

47 

60.55 

10.25 

43 

87.39 

8.50 

44 

87.19 

10.50 

45 

83.58 

7.75 

46 

75.78 

7.50 

47 

68.14 

7.00 

48 

77.84 

10.00 

49 

83.91 

4.25 

50 

81  .41 

10.75 

51 

75.27 

19.75 

52 

79.11 

11.75 

53 

89.87 

7.75 

54 

89.72 

8.25 

55 

79.93 

9.00 

56 

94.83 

6.25 

57 

74.26 

11.75 

58 

90 . 50 

7.50 

59 

84.18 

12.50 

B-46 


CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP14-2  LABEL  i  HYP  14  -  LEVEL  2  VAR  -  EPb  1,  2,  3,  it  12 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES;  2 


HYPOTHESIC  14  -  LEVEL  2  DIFF  -  CORR;  FINAL  SCORE  VB  TOTAL  TIME 


TAESCORE  TAE  TIME 
TAESCORE  1  .00000 
TAB  TIME  -.33604  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .21638 
CRITICAL  VALUE  (2-tai1 ,  .05)  =  +/-  .25614 

N  =  59 


B-47 


HYPOTHESIS  14  VAR  -  LEVEL  3  DIFF  -  TAE  FIN  SC  v»  TAE  TOTAL  TIME 


HEADER  DATA  FOR:  C»HYP14-3  LABEL*  HYP  14  -  LEVEL  3  VAR  -  EPISODES  4,11,14 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


TAESCORE 

TAE  TIME 

1 

72.03 

10.33 

o 

40.99 

33.00 

3 

79.70 

6.33 

4 

70.83 

22.33 

5 

70.57 

16.00 

6 

76.10 

9.67 

7 

64.37 

17.00 

8 

66.30 

8.33 

9 

80.87 

5.33 

1(0 

70.57 

16.33 

1 1 

52.70 

32.33 

12 

73.54 

14.33 

13 

44.61 

21  .33 

14 

38.20 

39.33 

15 

61  .50 

23.67 

16 

76.43 

11.67 

17 

53.81 

20.67 

18 

70.73 

23.33 

19 

72.62 

7.00 

20 

67.09 

12.67 

21 

52.33 

30.67 

22 

63.56 

7.67 

23 

66.04 

13.00 

24 

63.82 

17.33 

25 

35.61 

25.33 

26 

53.33 

18.67 

27 

69.87 

17.33 

28 

60.17 

36.00 

29 

70.97 

16.33 

30 

79.74 

15.33 

31 

56.05 

21.00 

32 

51  .66 

26.00 

33 

43.01 

22.00 

34 

36.79 

45.67 

35 

60.13 

11.00 

36 

46.30 

30.00 

37 

72.37 

18.00 

38 

85.43 

15.00 

39 

65.27 

20.67 

40 

77.83 

12.33 

41 

73.50 

11.33 

47 

71 .63 

14.33 

42 

76.13 

14.67 

44 

71 .83 

21.67 

45 

75.97 

13.00 

46 

72.94 

16.33 

47 

79.63 

10.33 

48 

69.84 

16.33 

49 

77.13 

18.33 

50 

57.77 

27.00 

51 

50.90 

29.00 

52 

72.10 

36.33 

53 

60.77 

28.67 

54 

83.10 

9.33 

55 

55.76 

22.33 

56 

87.50 

6.67 

57 

41  .77 

38.67 

58 

78.07 

14.67 

59 

54.21 

45.00 

B-48 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYPl4-3  LABEL:  HYP  14  -  LEVEL  3  VAR  -  EPISODES  4,11,14 
NUMBER  OF  CASES:  59  NUMBER  OF  V.'.RIABLES:  2 


HYPOTHESIS  14  -  LEVEL  3  DIFF  -  CORR:  FINAL  SCORE  vs  TOTAL  TIME 


TAESCORE  TAE  TIME 
TAESCDRF  1 .00000 
TAE  TIME  -.74653  1  .0000C} 


CRITICAL  VALUE  (1-TAIL,  .05)  =  ♦  Or  -  .21638 
CRITICAL  VALUE  (2-tail  ,  .05)  =  +/-  .25614 


HYPOTHESIS  14  VAR  -  LEVEL  4  DIFF  -  TAE  FIN  SC  v»  TAE  TOTAL  TIME 


HEADER  DATA  FOR:  C:HYP14-4  LABEL:  HYP  14  -  LEVEL  4  VAR  -  EPISODES  5  AND  13 
NUhttER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


TAESCORE 

TAE  TIME 

1 

37.26 

60.50 

2 

38.70 

75.50 

3 

70.91 

23.00 

4 

67.45 

18.50 

5 

59.80 

57.00 

6 

82.35 

24.50 

7 

78.44 

23.50 

8 

78.84 

11.50 

9 

71  .49 

13.00 

ICi 

68.75 

19.50 

1  1 

78.39 

12.00 

12 

77.10 

18.00 

13 

53.21 

20.00 

14 

62.89 

33.50 

15 

63.45 

19.50 

16 

91  .80 

12.00 

17 

70.95 

14.50 

18 

82.84 

13.50 

19 

69.20 

15.00 

20 

79.89 

12.00 

'21 

83.75 

20.50 

22 

55.15 

21.50 

23 

57.29 

60.00 

24 

79.49 

22.00 

25 

68.20 

14.00 

26 

47.60 

46.50 

27 

81  .99 

14.00 

28 

76.14 

33.00 

29 

39.95 

40.00 

30 

94.25 

9.50 

31 

58.85 

43.50 

32 

74.24 

12.50 

33 

75.39 

15.00 

34 

65.74 

31.50 

35 

49.99 

38.00 

36 

80.15 

30.00 

37 

73.64 

32.50 

38 

80.96 

25.00 

39 

72.08 

33.00 

40 

74.09 

18.00 

41 

63.09 

27.50 

4? 

77.84 

18.50 

4^ 

80.24 

10.50 

44 

55.37 

59.50 

45 

62.89 

26.50 

46 

76.94 

19.50 

47 

76.09 

16.00 

48 

58.10 

44.50 

49 

76.45 

23.50 

50 

39.01 

48.00 

51 

54  .04 

79.50 

52 

83.60 

18.00 

53 

61  .55 

38.50 

54 

83.60 

13.00 

55 

49.70 

82.00 

56 

83.29 

10.00 

57 

63.71 

66.00 

58 

84.40 

21 .00 

59 

64.75 

20.50 
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CORRELATION  MATRIX 


HEADER  DATA  FOR; 
NUMBER  OF  CASES; 


HYPOTHESIS  14  - 


TAESCORE 
TAE SCORE  1 .00000 
TAE  TIME  -.73553 


CRITICAL  VALUE  ( 
CRITICAL  VALUE  ( 

N  =  59 


C;HYP14-4  LABEL:  HYP  14  -  LEVEL  4  VAR  -  EPISODES  5  AND  13 
59  NUMBER  OF  VARIABLES;  2 


LEVEL  4  DIFF  -  CORR :  FINAL  SCORE  vs  TOTAL  TIME 

TAE  TIME 

1 .00000 

-TAIL,  .05)  =  +  Or  -  .2163B 
-tail  ,  .05)  =  */-  .25614 
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HYPOTHESIS  14  VAR  -  LEVEL  5  DIFF  -  TAE  FIN  SC  v*  TAE  TOTAL  TIME 


HEADER  DATA  FOR:  CiHYP14-5  LABEL:  HYP  14  -  LEVEL  5  VAR  -  EPISODES  6,  9,  10 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


TAESCORE 

TAE  TIME 

1 

62.25 

38.00 

2 

55.43 

19.67 

3 

54.50 

37.00 

4 

27.77 

57.33 

5 

45.58 

45.00 

6 

83.10 

20.33 

7 

50.15 

24.00 

8 

10.17 

71  .33 

9 

63.12 

20.00 

10 

39.88 

45.67 

1  1 

28.13 

62.67 

12 

43.36 

47.67 

13 

39.96 

38.00 

14 

28.30 

37.00 

15 

55.20 

17.33 

16 

38.69 

48.67 

17 

83.45 

6.67 

18 

56.10 

43.67 

19 

33.94 

39.00 

20 

44.56 

39.00 

21 

60.83 

24.00 

22 

36.76 

22.33 

23 

41  .73 

28.00 

24 

63.48 

36.33 

25 

57.28 

25.33 

26 

36.19 

47.33 

27 

50.03 

33.00 

28 

49.12 

60.33 

29 

62.73 

19.33 

30 

44.89 

52.00 

31 

55.79 

32.33 

32 

63.65 

10.00 

33 

61  .92 

10.67 

34 

44.37 

41 .33 

35 

36.92 

40.33 

36 

49.68 

31.33 

37 

46.93 

39.67 

38 

44.17 

67.67 

39 

58.06 

32.00 

40 

56.53 

21.33 

41 

52.39 

31 .67 

42 

40.56 

39.00 

43 

55.25 

42.33 

44 

42.90 

65.00 

45 

60.42 

16.33 

46 

36.86 

39.33 

47 

58.56 

35.67 

48 

51  .26 

25.67 

49 

66.43 

11.00 

50 

59.80 

21.33 

51 

56.50 

91 .00 

52 

47.25 

51.67 

53 

56.29 

46.33 

54 

50.46 

34.00 

55 

60.39 

28.00 

56 

53.86 

22.67 

57 

49.89 

65.00 

58 

59.80 

28.67 

59 

65.73 

24.00 
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-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP14-5  LABEL:  HYP  14  -  LEVEL  5  VAR  -  EPISODES  6,  9,  10 
NUMBER  wF  CASES;  59  NUMBER  OF  VARIABLES;  2 


HYPOTHECI*^  14  -  LEVEL  5  DIFF  -  CORR;  FINAL  SCORE  vs  TOTAL  TIME 


TAESCORE  TAE  TIME 
TAE SCORE  1  .00000 
TAE  TIME  -.5879B  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .2163B 
CRITICAL  VALUE  (2-tail ,  .05)  =  ♦/-  .25614 


N 


59 


HYPOTHESIS  15 
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DIFFICULTY  Z  SCORES  -  EPISODE  NUMBERS  IN  ORIGINAL  ORDER 


HEADER  DATA  FOR;  CtZSCORES  LABEL;  DIFF  Z  SCORES  -  EPs  IN  ORIGINAL  ORDER 
NUMBER  OF  CASES:  14  NUMBER  OF  VARIABLES;  2 


EPISODE 

Z  SCORES 

1 

2806.00 

-.81 

T 

A. 

2948.00 

-.77 

3 

2345.00 

-.94 

4 

4917.00 

-.19 

C 

w* 

8466 .00 

.85 

6 

9357 .00 

1.11 

7 

1625.00 

-1 .15 

B 

1686 .00 

-1  .14 

9 

9636 .00 

1  .20 

10 

12159 .00 

1  .94 

1  1 

5393.00 

-.05 

12 

3093.00 

-.72 

13 

8049.00 

.73 

14 

531 1 .00 

-.07 

HYPOTHESIS  15  VARIABLES 


EPISODE 

AVG  STUDENT  TIME 

AVG  INSTRUCTOR  TIME 

DIFFERENCE 

1 

6.96 

6.64 

.32 

2 

8.10 

7.18 

.92 

3 

11.46 

10.09 

1  .37 

4 

19.63 

22.27 

-2.64 

5 

21  .71 

22.00 

-  .29 

6 

23.50 

19.73 

3.77 

7 

3.71 

2.82 

.89 

8 

4.04 

3.27 

.77 

9 

40.77 

38.27 

2.50 

10 

43.71 

57.55 

-13.84 

11 

16.54 

21 .91 

-  5.37 

12 

9.10 

15.91 

-  6.81 

13 

31  .58 

54.36 

-22.78 

14 

19.60 

31 .09 

-11 .49 
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HYPOTHESIS  15  VARIABLES  -  Z  SCORES  St  TIME  DIFFERENCE  BET  S  8.  I 


HEADER  DATA  FOR;  CiHYPlS  LABEL;  HYP  15  VAR  -  Z  SCORES  St  TIME  DIFFERENCE 
NUMBER  OF  CASES:  14  NUMBER  OF  VARIABLES:  2 


Z  SCORE 

TIMEDIFF 

1 

.81 

.32 

2 

-.77 

.92 

3 

-.94 

1  .37 

4 

-.19 

-2.64 

5 

.85 

-.29 

6 

1.11 

3.77 

7 

-1 .15 

.89 

8 

-1 .14 

.77 

9 

1  .20 

2.50 

10 

1  .94 

-13.84 

11 

-.05 

-5.37 

12 

-.72 

-6.81 

13 

.73 

-22.78 

14 

-.07 

-11.49 

-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP15  LABEL:  HYP  15  VAR  -  Z  SCORES  It  TIME  DIFFERENCE 
NUMBER  OF  CASES;  14  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  15  -  CORR ;  Z  SCORES  vs  TIME  DIFFERENCE  BETWEEN  S  St  I 


Z  SCORE  TIMED IFF 
Z  SCORE  1 

TIMED IFF  -.34656  1  .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  ■  ♦  Or  -  .45900 
CRITICAL  VALUE  (2-t«il  ,  .05)  «  ♦/-  .53067 


HYPOTHESIS  17  VARIABLES  -  AVERAGE  TAE  SC  v«  AVERAGE  TOTAL  TIME 


HEADER  DATA  FOR:  ClHYP17  LABEL:  HYP  17  VAR  -  TAE  AVG  SCORE  v»  AVG  TIME 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


TAESCORE 

TIME 

1 

69.68 

22.57 

2 

60.88 

26.29 

3 

76.64 

14.14 

A 

70.83 

22.86 

5 

69.46 

26.93 

6 

81  .47 

13.93 

7 

70.49 

16.64 

8 

55.68 

22.07 

9 

75.45 

9.29 

lei 

70.15 

19.07 

1  1 

66.05 

23.57 

12 

73.57 

17.50 

13 

58.61 

17.93 

14 

56.83 

24.71 

15 

70.87 

12.93 

16 

73.49 

19.57 

17 

75.85 

10.07 

18 

75.54 

18.86 

19 

65.63 

16.50 

20 

69.18 

16.86 

21 

75.34 

16.64 

22 

61  .81 

14.79 

23 

66.73 

19.21 

24 

72.31 

17.14 

25 

67.35 

14.79 

26 

60.78 

24.07 

27 

76.54 

15.14 

28 

67.94 

29.36 

29 

66.67 

18.36 

30 

78.08 

18.50 

31 

68.05 

19.50 

32 

71  .01 

11.21 

33 

68.19 

11.36 

34 

66.80 

26.21 

35 

69.62 

20.93 

36 

65.84 

21.93 

37 

72.05 

19.86 

38 

75.89 

24.86 

39 

74.63 

18.79 

40 

76.99 

12.50 

41 

74.49 

15.79 

42 

71  .82 

17.43 

4  - 

78.35 

16.93 

44 

68.74 

32.21 

45 

76.29 

12.36 

46 

70.05 

17.50 

47 

79.77 

14.50 

48 

70.35 

18.71 

49 

79.74 

11.00 

50 

67.65 

20.57 

51 

66.20 

42.93 

52 

74.15 

25.21 

53 

73.51 

24.36 

54 

80.18 

13.93 

55 

68  .85 

25.43 

56 

81  .39 

10.29 

57 

62.35 

35.50 

58 

81  .42 

15.00 

59 

72.20 

21.79 
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CORRELATION  MATRIX 


HEADER  DATA  FOR:  C:HYP17  LABEL:  HYP  17  VAR  -  TAE  AVG  SCORE  vs  AVG  TIME 
NUMBER  OF  CASES:  59  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  17  -  CORR :  AVERAGE  TAE  SCORE  vs  AVERAGE  TOTAL  TIME 


TAESCORE  TIME 

TAESCQRE  1 

TIME  -.49233  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  -*-  Or  -  .21638 
CRITICAL  VALUE  v2-tail ,  .05)  =  +/-  .25614 

N  =  59 


B-59 


HYPOTHESIS  18 


B-60 


HYPOTHESIS  18  VARIABLES  -  TIME  TILL  1ST  REF  DESIGN/DIAG  TEST 


HEADER  DATA  FOR:  CiHYPlB  LABEL  i  HYP  18  VAR  -  TIME  TILL  1ST  REF/DIAG  TEST 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES:  2 


TAESCORE 

TIMETILL 

1 

69.68 

2.11 

2 

60.88 

4.14 

3 

76.64 

3.04 

4 

70.83 

4.11 

5 

69.46 

2.29 

6 

81  .47 

1  .64 

7 

70.49 

2.36 

8 

55.88 

1 .57 

9 

75.45 

1  .00 

1<? 

70.15 

3.61 

1  1 

66.05 

1  .71 

12 

73.57 

2.11 

13 

58.61 

1  .68 

14 

56.83 

1 .71 

15 

70.87 

1.18 

16 

73.49 

3.46 

17 

75.85 

.36 

18 

75.54 

1 .25 

19 

65.63 

.29 

20 

69.18 

2.36 

21 

75.34 

2.07 

22 

61  .81 

1.25 

23 

66.73 

1 .82 

24 

72.31 

3.50 

25 

67.35 

1.25 

26 

60.78 

6.18 

27 

76.54 

2.50 

28 

67.94 

6.14 

29 

66.67 

2.64 

30 

78,08 

1 .75 

31 

68.05 

2.54 

32 

71  .01 

1  .50 

33 

68.19 

.93 

34 

66.80 

4.43 

35 

69.62 

2.57 

36 

65.84 

4.46 

37 

72.05 

2.57 

38 

75.89 

2.79 

39 

74.63 

3.39 

40 

76.99 

1.21 

41 

74.49 

4.68 

47 

71  .82 

1.79 

4  J 

78.35 

1  .89 

44 

68.74 

4.82 

45 

76.29 

1.68 

46 

70.05 

3.04 

47 

79.77 

2.32 

48 

70.35 

1  .64 

49 

79.74 

1 .57 

50 

67.65 

3.57 

51 

66.20 

3.04 

52 

74.15 

1 .75 

53 

73.51 

2.75 

54 

80.18 

1 .86 

55 

68.85 

2.21 

56 

81  .39 

2.61 

57 

62.35 

5.29 

58 

81  .42 

1 .61 

59 

72.20 

2.00 
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CORRELATION  MATRIX 


HEADER  DATA  FOR:  C.-HYPIB  LABEL:  HYP  18  VAR  -  TIME  TILL  1ST  REF/DI  AO  TEST 
NUMBER  or  CASES:  59  NUMBER  OF  VARIABLES:  2 


HYP  IB  TEST  -  CORR:  TAE  FINAL  SC  vs  TIME  TILL  1ST  REF/DIAG  TEST 


TAESCORE  TIMETILL 
TAESCORE  1 .00000 
TIMETILL  -.23814  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  +  Or  -  .21638 
CRITICAL  VALUE  (2-tail ,  .05)  =  +/-  .25614 

N  =  59 
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HYPOTHESIS  19 


B-63 


HYP  19  VAR  -  TAE  AVERAGE  SCORE  v*  «  OF  O-SCOPE  TESTS  -  S  8<  I 


HEADER  DATA  FOR;  CiHYP19  LABEL;  HYP  19  VAR  -  TAE  FINAL  SCORE  v»  O-SCOPE 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES;  2 


I 

I 


TAESCORE 

O-SCOPE 

1 

69.68 

5.43 

2 

60.ee 

.21 

3 

76.64 

3.29 

4 

70.83 

4.79 

5 

69.46 

4.93 

6 

81  .47 

2.21 

7 

70.49 

1 .43 

8 

55.88 

3.93 

9 

75.45 

1 .29 

10 

70.15 

1 .79 

1  1 

66.05 

4.00 

12 

73.57 

2.93 

13 

58.61 

2.21 

14 

56.83 

1 .86 

15 

70.87 

3.57 

16 

73.49 

3.79 

17 

75.85 

.29 

18 

75.54 

4.36 

19 

65.63 

1.07 

20 

69.18 

.93 

21 

75.34 

6.50 

22 

61  .81 

.14 

23 

66.73 

.57 

24 

72.31 

1.71 

25 

67.35 

1.50 

26 

60.78 

.64 

27 

76.54 

2.21 

28 

67.94 

4.64 

29 

66.67 

.64 

30 

78.08 

4.14 

31 

68.05 

1.00 

32 

71  .01 

.64 

33 

68.19 

.50 

34 

66.80 

3.36 

35 

69.62 

2.86 

36 

65.84 

1.21 

37 

72.05 

6.86 

38 

75.89 

8.36 

39 

74.63 

5.14 

40 

76.99 

1.29 

41 

74.49 

2.14 

42 

71 .82 

1.50 

43 

78.35 

2.43 

44 

68.74 

6.36 

45 

76.29 

1.64 

46 

70.05 

3.14 

47 

79.77 

2.29 

48 

70.35 

6.00 

49 

79.74 

1.64 

50 

67.65 

.93 

51 

66.20 

3.64 

52 

74.15 

5.79 

53 

73.51 

7.86 

54 

80.18 

1.71 

55 

68.85 

2.00 

56 

81  .39 

1.14 

57 

62.35 

6.93 

58 

81  .42 

7.71 

59 

72.20 

2.00 
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CORRELATION  HATRIX 


HEADER  DATA  FOR;  C:HYP19  LABEL:  HYP  19  VAR  -  TAE  FINAL  SCORE  vs  O-SCOPE 
NUMBER  OF  CASES;  59  NUMBER  OF  VARIABLES;  2 


HYPOTHESIS  19  TEST  -  CORRELATION:  TAE  FINAL  SC  vs  #  OSCOPE  TESTS 


TAESCORE  O-SCOPE 
TAESCORE  1 

O-SCOPE  .16771  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  =  ♦  Or  -  .21638 
CRITICAL  VALUE  (2-tail,  .05)  =  +/-  .25614 

N  =  59 


B-65 


HYPOTHESIS  20 


B-66 


HYPOTHESIS  20  -  CLASS  4/14  -  TAE  RANK  vb  INSTRUCTOR  RANK 


HEADER  DATA  FOR:  C;HYP20-4  LABEL:  HYP  20  -  4/14  -  TAE  RANK  vs  INSTR  RANK 
NUHBLR  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


TAERANK  INSTRANK 


1  5.00  5.00 

2  7.00  7.00 

3  2.00  1.00 

4  3.00  3.00 

5  6.00  6.00 

6  1.00  2.00 

7  4.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  CiHYP20-4  LABEL:  HYP  20  -  4/14  -  TAE  RANK  vb  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  20  -  CLASS  4/14  -  CORRELATION:  TAE  RANK  vb  INSTR  RANK 


TAERANK  INSTRANK 
TAERANK  1 .00000 
INSTRANK  .96429  1 .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .67649 
CRITICAL  VALUE  (2-t«il  ,  .05)  «  •*•/-  .75315 

N  -  7 


B-67 


HYPOTHESIS  20  -  CLASS  5/12  -  TAE  RANK  vb  INSTRUCTOR  RANK 


HEADER  DATh  FOR:  CiHYP20-5  LABEL:  HYP  20  -  5/12  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


TAE  RANK  INSTRANK 


1  7.00  4.00 

2  1.00  1.00 

3  3.00  3.00 

4  4.00  7.00 

5  2.00  5.00 

6  5.00  2.00 

7  6.00  6.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP20-5  LABEL:  HYP  20  -  5/12  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  20  -  CLASS  5/12  -  CORRELATION:  TAE  RANK  vs  INSTR  RANK 


TAE  RANK  INSTRANK 
TAE  RANK  1 .00000 
INSTRANK  .35714  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .67649 
CRITICAL  VALUE  (2-t«il ,  .05)  -  ♦/-  .75315 

N  «  7 


B-68 


HYPOTHESIS  2®  -  CLASS  6/9  -  TAE  RANK  vb  INSTRUCTOR  RANK 


HEADER  DATA  FOR:  C:HYP2®-6  LABEL:  HYP  2®  -  6/9  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


TAERANK 

INSTRANK 

1 

5.®® 

3.®® 

o 

4  .®® 

2.®® 

3 

1  .®0 

1  .®0 

4 

2.®® 

6.00 

5 

7.®0 

5.®0 

6 

6  .®® 

7,<> 

7 

3.®® 

4  .vt 

-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP2®-6  LABEL:  HYP  2®  "•  6/9  -  TAE  RANK  vb  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  2®  -  CLASS  6/9  - 

CORRELATION 

:  TAE  RANK  vb  INSTR  RANK 

TAERANK  INSTRANK 

TfiERANK 

1  .®®®®® 

iNSTRANK 

.46429  1  .®®®®® 

CRITICAL 

VALUE  (1-TAIL,  .®5) 

»  Or  - 

.67649 

CRITICAL 

value  (2-t«i1 ,  .«5) 

«  ♦/- 

.753X5 

N  -  7 

B-69 


hypothesis  20  -  CLASS  7/7  -  TAE  RANK  vs  INSTRUCTOR  RANK 


HEADER  DATt,  FOR:  C:HYP20-7  LABEL:  HYP  20  -  7/7  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  6  NUMBER  OF  VARIABLES:  2 


TAE  RANK  INSTRANK 

1  5.00  4.00 

2  4.00  1.00 

3  2.00  2.00 

4  3.00  5.00 

5  6.00  6.00 

6  1.00  3.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP20-7  LABEL:  HYP  20  -  7/7  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  6  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  20  -  CLASS  7/7  -  CORRELATION:  TAE  RANK  vs  INSTR  RANK 


TAE  RANK  INSTRANK 
TAE  RANK  1 .00000 
INSTRANK  .48571  1 .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  +  Or  -  .73972 
CRITICAL  VALUE  <2-t«il ,  .05)  -  ♦/-  .81165 

N  «  6 


B-70 


HYPOTHESIS  20  -  CLASS  8/4  -  TAE  RANK  vs  INSTRUCTOR  RANK 


HEADER  DAT(^  FOR:  C:HYP20-8A  LABEL:  HYP  20  -  8/4  -  TAE  RANK  vs  INSTR  RANK 
NUHBER  OP  CASES:  6  NUMBER  OF  VARIABLES;  2 


TAE  RANK  INSTRANK 

1  5.00  2.00 

2  6.00  5.00 

3  1.00  6.00 

4  4.00  4.00 

5  2.00  3.00 

6  3.00  1.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR;  C;HYP20-8A  LABEL:  HYP  20  -  8/4  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES;  6  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  20  -  CLASS  8/4  -  CORRELATION:  TAE  RANK  vs  INSTR  RANK 


TAE  RANK  INSTRANK 
TAE  RANK  1 .00000 
I NSTRANK  - . 1 4286  1  .00000 


critical  value  <1-TAIL,  .05)  *  +  Or  -  .73972 
CRITICAL  VALUE  (2-t«il  ,  .05)  -  */-  .81165 

N  -  6 


B-71 


HYPOTHESIS  20  -  CLASS  B/25  -  TAE  RANK  vs  INSTRUCTOR  RANK 


HEADER  DATA  FOR:  C:HYP20-8B  LABEL:  HYP  20  -  8/25  -  TAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  8  NUMBER  OF  VARIABLES:  2 


TAE  RANK 

INSTRANK 

1 

7,00 

6  .00 

2 

6.00 

7  .00 

3 

8.00 

4  .00 

4 

5.00 

3.00 

5 

2.00 

5.00 

6 

3  .00 

2.00 

7 

1  .00 

8.00 

e 

4 .00 

1  .00 

CORRELATION  MATRIX 


HEADER  DATA  FOR;  C:HYP20-8B  LABEL;  HYP  20 
NUMBER  OF  CASES;  8  NUMBER  OF  VARIABLES;  2 


8/25  -  TAE  RANK  vs  INSTR  RANK 


HYPOTHESIS  20  -  CLASS  8/25  -  CORRELATION:  TAE  RANK  vs  INSTR  RANK 


TAE  RANK  INSTRANK 
TAE  RANK  1 .00000 
INSTRANK  -.07143  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  Or  -  .62658 
CRITICAL  VALUE  (2-t«il  ,  .05)  ■  ♦/-  .70477 

N  «  8 


B-72 


HYPOTHESIS  20  -  CLASS  9/29  -  TAE  RANK  vb  INSTRUCTOR  RANK 


HEADER  DATA  FOR:  C:HYP20-9  LABEL:  HYP  20  -  9/29  -  VAE  RANK  vs  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES;  2 


TAERANK  INSTRANK 
4 .00  4 .00 

2.00  2.00 

7.00  7.00 

3.00  3.00 

6.00  5.00 

1.00  1.00 

5.00  6.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  ClHYP20-9  LABEL:  HYP  20  -  9/29  -  TAE  RANK  vb  INSTR  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  20  -  CLASS  9/29  -  CORRELATION:  TAE  RANK  vb  INSTR  RANK 


TAERANK  INSTRANK 
TAERANK  1 .00000 
INSTRANK  .96429  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .67649 
CRITICAL  VALUE  <2-t«i1  ,  .05)  -  ■•■/"  .75315 

N  -  7 


B-73 


HYPOTHESIS  21 


B-74 


HYPOTHESIS  21  -  CLASS  4/14  -  TAE  RANK  vs  CLASS  RANK 


HEADER  DATA  FDR:  C:HYP21-4  LABEL:  HYP  21  -  4/14  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  7  NUMBER  OF  VARIABLES:  2 


TAERANK  CLASRANK 

1  5.00  5.00 

2  7.00  7.00 

3  2.00  1.00 

4  3.00  2.00 

5  6.00  6.00 

6  1.00  3.00 

7  4.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  CiHYP21-4  LABEL:  HYP  21  -  4/14  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  21  -  CLASS  4/14  -  CORRELATION:  TAE  RANK  vs  CLASS  RANK 


TAERANK  CLASRANK 
TAERANK  1 .00000 
CLASRANK  .69286  1 .00000 


CRITICAL  VALUE  <1-TAIL,  .05)  «  ♦  Or  -  .67649 
CRITICAL  value  <2-t«il  ,  .05)  -  ♦/-  .75315 


HYPO^'SESIS  21  -  CLASS  5/12  -  TAE  RANK  vs  CLASS  RANK 


HEADER  data  FOR;  C:HYP21-5  LABEL;  HYP  21  -  5/12  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  7  NUMBER  OF  VARIABLES;  2 


TAE  RANK  CLASRANK 

1  7.00  5.00 

2  1.00  3.00 

3  3.00  1.00 

4  4.00  6.00 

5  2.00  2.00 

6  5.00  7.00 

7  6.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR;  C;HYP21-5  LABEL;  HYP  21  -  5/12  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  7  NUMBER  OF  VARIABLES;  2 


HYPOTHESIS  21  -  CLASS  5/12  -  CORRELATION;  TAE  RANK  vs  CLASS  RANK 


TAE  RANK  CLASRANK 
TAE  RANK  1 .00000 
CLASRANK  .57143  1.00000 


CRITICAL  VAUUE  < 1-TAIL,  .05)  -  ♦  Or  -  .67649 
critical  VALUE  (2-t«il ,  .05)  «  ♦/-  .75315 

N  «  7 


B-76 


HYPOTHESIS  21  -  CLASS  6/9  -  TAE  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  CiHYP21-6  LABEL:  HYP  21  -  6/9  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


TAERANK  CLASRANK 


1  5.00  5.00 

2  4.00  1.00 

3  1.00  7.00 

4  2.00  3.00 

5  7.00  6.00 

6  6.00  2.00 

7  3.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  CiHYP21-6  LABEL:  HYP  21  -  6/9  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  21  -  CLASS  6/9  -  CORRELATION:  TAE  RANK  vs  CLASS  RANK 


TAERANK  CLASRANK 
TAERANK  1 .00000 
CLASRANK  -.14286  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  ■  ♦  Or  -  .67649 
CRITICAL  VA..UE  (2-t«il  ,  .05)  *  ♦/-  .75315 

N  ■  7 


B-77 


hypothesis  21  -  CLASS  7/7  -  TAE  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR;  CiHYP21-7  LABEL:  HYP  21  -  7/7  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  DR  CASES:  6  NUMBER  OF  VARIABLES;  2 


1 

O 

3 

4 

5 

6 


TAE  RANK 
5.00 
4  .00 
2.00 
3.00 
6  .00 
1  .00 


CLASRANK 
5.00 
1  .00 
4.00 
2.00 
6.00 
3.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR;  CiHYP21-7  LABEL:  HYP  21  -  7/7  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  6  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  21  -  CLASS  7/7  -  CORRELATION:  TAE  RANK  vs  CLASS  RANK 


TAE  RANK  CLASRANK 
TAE  RANK  1  .00000 
CLASRANK  .4B571  1 .00000 


CRITICAL  VALUE  <1-TAIL,  .05)  -  +  Or  -  .73972 
CRITICAL  VALUE  <2-tail ,  .05)  «  ♦/-  .81 165 

N  -  6 


B-78 


HYPOTHESIS  21  -  CLASS  8/4  -  TAE  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C;HYP21-8A  LABEL:  HYP  21  -  8/4  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  6  NUMBER  OF  VARIABLES:  2 


TAE  RANK  CLASRANK 


1  5.00  3.00 

2  6 . 00  5 . 00 

3  1 .00  1 .00 

4  4.00  2.00 

5  2.00  4.00 

6  3.00  6.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C*HYP21-8A  LABEL:  HYP  21  -  8/4  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  6  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  21  -  CLASS  8/4  -  CORRELATION:  TAE  RANK  vs  CLASS  RANK 


TAE  RANK  CLASRANK 
TAE  RANK  1 .00000 
CLASRANK  .37143  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .73972 
CRITICAL  VALUE  <2-tail,  .05)  «=  ♦/-  .81165 

N  -  6 


B-79 


HYPOTHESIS  21  -  CLASS  8/25  -  TAE  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C:HYP21-8B  LABEL:  HYP  21  -  8/25  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  8  NUMBER  OF  VARIABLES:  2 


TAE  RANK  CLASRANK 


1  7.00  3.00 

2  6.00  8.00 

3  8.00  1.00 

4  5.00  2.00 

5  2.00  6.00 

6  3.00  5.00 

7  1.00  7.00 

8  4.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  CiHYP21-8B  LABEL:  HYP  21  -  8/25  -  TAE  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  8  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  21  -  CLASS  8/25  -  CORRELATION:  TAE  RANK  v*  CLASS  RANK 


TAE  RANK  CLASRANK 
TAE  RANK  1 .00000 
CLASRANK  -.59524  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .62658 
CRITICAL  VA..UE  (2-t«ii1  ,  .05)  «  ♦/-  .70477 


HYPOTHESIS  21  -  CLASS  9/29  -  TAE  RANK  vb  CLASS  RANK 


HEADER  DATA  FOR:  C:HYP21-9  LABEL:  HYP  21  -  9/29  -  TAE  RANK  vs  CLASS  RANK 
number  op  CASES;  7  NUMBER  OF  VARIABLES:  2 


1 

2 

3 

A 

5 

6 
7 


TAERANK 
4.00 
2.00 
7.00 
3.00 
6  .00 
1  .00 
5.00 


CLASRANK 
7.00 
3.00 
4  .00 
2.00 
5.00 
1  .00 
6.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP21-9  LABEL:  HYP  21  -  9/29  -  TAE  RANK  vb  CLASS  RANK 
NUMBER  OF  CASES;  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  21  -  CLASS  9/29  -  CORRELATION;  TAE  RANK  vb  CLASS  RANK 


TAERANK  CLASRANK 
TAERANK  1 .00000 
CLASRANK  .60714  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  «  •»  Or  -  .67649 
CRITICAL  VALUE  (2-t«il ,  .05)  -  ♦/-  .75315 


HYPOTHESIS  22 


> 


2 


B-82 


HYPOTHESIS  22  -  CLASS  4/14  -  INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C;HYP22-4  LABEL:  HYP  22  -  4/14  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


INSTRANK  CLASRANK 

1  5.00  5.00 

2  7.00  7.00 

3  1 .00  1  .00 

4  3.00  2.00 

5  6.00  6.00 

6  2.00  3.00 

7  4.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C  :HYP22-4  LABEL:  HYP  22  -  4/14  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  22  -  CLASS  4/14  -  CORRELATION;  INS  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .00000 
CLASRANK  .96429  1 .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  »  +  Or  -  .67649 
CRITICAL  VALUE  (2-t«il  ,  .05)  *  ♦/-  .75315 

N  -  7 


B-83 


HYPOTHESIS  22  -  CLASS  5/12  -  INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C;HYP22-5  LABEL:  HYP  22  -  5/12  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  8  NUMBER  OF  VARIABLES:  2 


INSTRANK  CLASRANK 

1  5 .00  5  ^00 

2  2.00  3.00 

3  4.00  1.00 

4  8.00  7.00 

5  6.00  2.00 

6  3.00  8.00 

7  1 .00  6 .00 

B  7.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP22-5  LABEL:  HYP  22  -  5/12  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  8  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  22  -  CLASS  5/12  -  CORRELATION:  INS  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .00000 
CLASRANK  -.02381  1.00000 


CRITICAL  VALUE  < 1-TAIL,  .05)  ■  ♦  Or  -  .62658 
CRITICAL  VALUE  (2-tai1  ,  .05)  «  ♦/-  .70477 

N  *  8 


HYPOTHESIS  22  -  CLASS  6/9  -  INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C:HYP22-6  LABEL;  HYP  22  -  6/9  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  7  NUMBER  OF  VARIABLES;  2 


INSTRANK  CLASRANK 

1  3.0C)  5.00 

2  2.00  1.00 

3  1.00  7.00 

4  6.00  3.00 

5  5.00  6.00 

6  7.00  2.00 

7  4.00  4.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  CiHYP22-6  LABEL;  HYP  22  -  6/9  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  7  NUMBER  OF  VARIABLES;  2 


HYPOTHESIS  22  -  CLASS  6/9  -  CORRELATION:  INST  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .00000 
CLASRANK  -.35714  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  «  ♦  Or  -  .67649 
critical  value  (2-tail,  .05)  «  */-  .75315 

N  -  7 


B-85 


HYPOTHESIS  22  -  CLASS  7/7 


INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C:HYP22-7  LABEL:  HYP  22  -  7/7  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


INSTRANK  CLASRANK 

1  4.00  5.00 

2  6.00  6.00 

3  1 .00  1 .00 

4  2.00  4.00 

5  5.00  2.00 

6  7.00  7.00 

7  3.00  3.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP22-7  LABEL:  HYP  22  -  7/7  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  22  -  CLASS  7/7  -  CORRELATION:  INST  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .00000 
CLASRANK  .75000  1 .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  «  ♦  Or  -  .67649 
CRITICAL  VA1.LIE  (2-t«il  ,  .05)  «  ♦/-  .75315 

N  «  7 


B-86 


HYPOTHEGIS  22  -  CLASS  8/4  -  INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR: 
NUMBER  OF  CASES; 


C:HYP22-8A  LABEL:  HYP  22 
8  NUMBER  OF  VARIABLES:  2 


8/4 


INSTR  RANK  vs  CLASS  RANK 


INSTRANK 

CLASRANK 

1 

7.00 

7.00 

2 

8.00 

8.00 

3 

2.00 

3.00 

4 

5.00 

5.00 

5 

6  .00 

1  .00 

6 

4  .00 

2.00 

7 

3.00 

^  .00 

8 

1  .00 

6.00 

-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR;  CiHYP22-8A  LABEL:  HYP  22  -  B/4  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  8  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  22  -  CLASS  B/4  -  CORRELATION:  INST  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .0000® 

CLASRANK  .33333  1.00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  +  Or  -  .62658 
CRITICAL  VAi_UE  (2-t«il  ,  .05)  «  ♦/-  .70477 


N  >  8 
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HYPOTHESIS  22  -  CLASS  8/25  -  INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR;  C;HYP22-BB  LABEL;  HYP  22  -  8/25  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  9  NUMBER  OF  VARIABLES:  2 


INSTRANK 

CLASRANK 

1 

7.00 

6.00 

o 

A. 

6.00 

3.00 

3 

8.00 

9.00 

4 

4  .00 

1  .00 

5 

3.00 

2-00 

6 

5.00 

7.00 

7 

2.00 

5.00 

8 

9.00 

8.00 

9 

1  .00 

4.00 

-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR;  CiHYP22-8B  LABEL;  HYP  22  -  8/25  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES;  9  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  22  -  CLASS  8/25  -  CORRELATION;  INS  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .Beeee 
CLASRANK  .63333  1  .00900 


CRITICAL  VALUE  (1-TAIL,  .05)  »  ♦  Or  -  .58607 
CRITICAL  VALUE  (2-t«i1,  .05)  -  ♦/-  .66422 


HYPOTHESIS  22  -  CLASS  9/29  -  INSTRUCTOR  RANK  vs  CLASS  RANK 


HEADER  DATA  FOR:  C:HYP22-9  LABEL:  HYP  22  -  9/29  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


INSTRANK  CLASRANK 


1  4.00  7.00 

2  2.00  3.00 

3  7.00  4.00 

4  3.00  2.00 

5  5.00  5.00 

6  1  .00  1 .00 

7  6.00  6.00 


-  CORRELATION  MATRIX  - 

HEADER  DATA  FOR:  C:HYP22-9  LABEL:  HYP  22  -  9/29  -  INSTR  RANK  vs  CLASS  RANK 
NUMBER  OF  CASES:  7  NUMBER  OF  VARIABLES:  2 


HYPOTHESIS  22  -  CLASS  9/29  -  CORRELATION:  INS  RANK  vs  CLASS  RANK 


INSTRANK  CLASRANK 
INSTRANK  1 .00000 
CLASRANK  .64286  1 .00000 


CRITICAL  VALUE  (1-TAIL,  .05)  -  ♦  Or  -  .67649 
CRITICAL  value  (2-t«il ,  .05)  ■  ♦/-  .75315 

N  «  7 
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APPENDIX  C 

FACTOR  RANKING  QUESTIONNAIRE 
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NAVY  PERSONNEL  RESEARCH  AND  DEVELOPHENT  CENTER 
TRODBLESEOOTING  FACTORS  BVALDATION  QUESTIONNAIRE 


INTRODUCTION 

The  Chief  of  Naval  Operations  (via  the  Vice  Chief  of  Naval  Operations 
for  Manpower,  Personnel  and  Training)  has  stated  a  requirement  of 
developing  a  reliable  and  objective  method  for  evaluation  of  the 
troubleshooting  skills  of  Navy  technicians.  The  Navy  Personnel  Research 
and  Development  Center,  San  Diego,  California  has  been  tasked  to  collect 
information  and  carry  out  this  requirement.  The  name  of  the  research 
and  development  project  that  is  accomplishing  this  tasking  is  the 
■Troubleshocting  Proficiency  Evaluation  Program  (TPEP)." 

You  have  been  selected  to  participate  in  this  effort  because  you  are 
considered  a  Subject  Matter  Expert  (SME)  in  troubleshooting  Navy  systems 
and  equipment.  The  information  that  you  provide  will  be  valuable  and 
will  be  used  as  the  basis  for  developing  a  method  of  evaluating  the 
troubleshooting  skills  of  Navy  personnel. 

Please  keep  in  mind,  that  this  is  not  in  any  wa^  a  test,  nor  will  the 
information  you  provide  be  released  to  persons  other  than  yourself  or 
those  directly  concerned  with  the  project.  You  or  your  career  will  not 
be  jeopardized  in  any  way. 

GENERAL  INSTRUCTIONS 

.  Please  read,  date,  provide  SSN  and  sign  "Privacy  Act  Statement" 
provided  below. 

2.  Please  read  the  instructions  which  precede  each  of  the  two  parts  of 
the  questionnaire  carefully  before  responding  to  any  items. 

3.  If  you  wish  to  change  a  response,  be  sure  to  completely  erase  (or 
clearly  mark  out  if  using  pen)  any  previous  response. 


PRIVACY  ACT  STATEMENT 

You  have  been  selected  to  participate  in  the  TPEP  project.  This  project 
provides  research  data  on  the  different  levels  of  troubleshooting 
expertise  associated  with  Navy  systems  and  equipment.  The  information 
provided  by  you  will  be  used  by  the  Navy  Personnel  Research  and 
Development  Center,  San  Diego,  for  research  purposes  only.  It  will  not 
become  a  part  of  your  official  record,  nor  will  it  be  used  to  make 
decisions  about  you  which  will  affect  your  career  in  any  way.  Your 
name,  SSN  are  necessary  only  to  aid  in  processing  the  research  data. 

Date:  _ 


SSN: 


ignatu  r e : 
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SPECIAL  INSTRDCTIONS  PART  I 
PART  I  -  BACKGROOWD  INFORMATION 

I'he  items  in  Part  I  concern  your  general  background.  Unless  a  written 
response  is  required,  please  place  an  "X"  in  the  box  which  corresponds 
to  the  answer  you  have  selected. 


BACKGROUND  ITEMS 


1.  _ ,  _  _ 

Last  Name  First  Name  H.I. 


2. _ - _ - _ _ 

Social  Security  Number 

3.  _ _ _ 

Job  Title  (e.g.,  Div.  Chief;  Work  Center  Supvr.;  Instructor,  etc.,) 

4.  _ 

Rate  &  Rating  or  Rank 

5.  _ 

Primary  NEC  (current) 

\ 

o . _ 

Ship  or  Station  and  Department/Division 

7.  My  highest  educational  level  is  best  described  as  (  x  one  please) 

_  Some  High  School 

_  High  School  Diploma 

_  Some  college 

_  Associate's  degree 

_  Bachelor's  degree 

_  Some  graduate  work 

_  Master's  degree 

_  Post  master's  work 

_  Doctorate  degree 


_ _ 

Major  area  of  formal  education 
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rhe  cards  that  are  supplied  in  Part  li  of  the  questionnaire  concern 
factors  related  to  the  evaluation  of  troubleshooting  skills.  Each  card 
contains  a  description  of  one  evaluation  factor  (with  some  example 
information  on  the  factor).  Please  read  All  the  following  information 
and  each  of  the  "factor"  cards  before  proceeding  with  the  sorting 
instruction"^  provided  on  the  next  page. 

LIMITATION  OP  THE  TRODBLESHOOTIWG  FACTORS 

It  is  understood  that  there  are  a  number  of  factors  that  are  involved  in 
evaluating  a  troubleshooters  capability  and  that  these  factors  vary  in 
importance  (i.e.,  they  have  different  weights).  In  this  "ranking"  the 
factors  have  been  restricted,  intentionally^  for  varying  reasons  (e.g., 
safety  for  personnel  and  hardware  are  not  included  for  it  is  understood 
that  if  you  violate  safety  you  are  a  "bad"  troubleshooter).  Also,  use 
of  test  equipment,  soldering,  and  other  "support*  skills  are  n.^t 
included.  These  are  considered  support  and  can  be  measured  via  other 
methods.  "Troubleshooting,"  however,  is  considered  to  be  a  mental  effort 
and  factors  are  required  that  can  be  checked  to  see  if  the  mental 
process  of  one  troubleshooter  is  more  efficient  than  another.  Figure  1 
shows  the  factors  that  you  are  being  asked  to  rank  order  in  the  order  of 
importance . 

"'.QNDITIONS  OP  TRODBLESBOOTING  EPISODE 

in  selecting  and  "ranking"  of  factors  that  one  would  use  in  determining 
(evaluating)  if  one  troubleshooter  was  more  efficient  than  another  it  is 
necessary  that  the  environment  (e.g.,  war  or  peace,  at  sea  or  tied  up, 
etc.)  of  the  troubleshooting  effort  be  identified.  If  the  environment 
changes  this  may  change  the  ranking  results  . 

For  purposes  of  this  selection  and  ranking  effort  assume  the  following 
environment/conditions : 

a.  NON-COMBAT  (i.e.  peace  time) 

b.  NORMAL  DAY  IN  HOME-PORT 

c.  DURING  A  DAILY  CHECK  OF  THE  EQUIPMENT  A  TROUBLE  IS  INDICATED 


TEST  EQUIPMENT  USE 


OTHER  SUPPORT  SKILLS 


«  OF  OUT  OF  BOUNDS 


FOLLOWED  PROCEDURES. 


#  OF  ILLOGICAL  APPROACHES 


TPEP 


i  OF  INVALID  CHECKS 


#  OF  PROOF  POINTS. 


TIME 


:cosT 


SOLUTION 


FIGURE  1.  TROUBLESHOOTING  FACTORS  TO  BE  RANKED 


fter  reading  the  information  above  and  reviewing  all  the  factor  cards 
please  perform  the  following  steps; 

1.  Sort  the  cards  according  to  how  important  the  factor  on  a  card  is  in 
evaluating  troubleshooting  ability. 

2.  Once  all  the  cards  are  sorted  in  a  sequence,  circle  the  number  on 
the  card  that  corresponds  to  the  cards  position  in  the  sequence  (e.g.. 
Most  Important  is  card  1;  Second  Most  Important  is  2;  and  so-on  until 
you  get  to  the  last  card  which  will  be  number  10  and  the  least 
important) . 

NOTE 

Make  certain  you  have  circled  the  appropriate  number  for  each  card 
(located  in  the  upper  right  hand  corner  of  each  factor  card). 

Circle  a  numl^er  only  once  for  each  factor  should  have  a  different  level 
of  importance. 

Circle  only  one  number  per  card. 

After  completing  the  questionnaire  please  place  all  the  material  into 
the  envelope  provided  and  mail. 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
^  Most  -  -  Medium  -  -  Least 

(circle  1  per  card} 

Factor  Title:  FOLLOW  PROCEOORES 


Factor  Description/Definition:  When  given  a  troubleshooting  or 

maintenance  procedure  the  troubleshooter 
follows  the  directions  as  presented. 


Example  of  Factor  Application:  The  person  troubleshooting  a  system  when 

provided  fault  check  or  test  procedures 
does  them  as  directed. 


4  ;WiJ  :j  n  #  i  4  J  !•#  1 3}  iS  4*/.^  *  f  •}  v  ■  l>  ^ 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
Most  -  -  Medium  -  -  Least 
(circle  1  per  card) 

Factor  Title:  NUMBER  OF  CHECKS  HADE 


Factor  Description/Definition:  Number  of  checks  performed  to  isolate 

the  fault.  These  include  continuity, 
logic,  frequency,  current,  voltage  and 
waveforms. 

Example  of  Factor  Application:  The  troubleshooter  does  not  make  a 

number  of  different  tests  (logic, 
voltage,  waveform  etc.)  at  the  same  test 
point  using  different  test  equipment 
when  one  test  provides  the  information 
needed. 


QN_PAQGBAM  (TPEP^ 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
Kost  -  -  Medium  -  -  Least 
(circle  1  per  card) 


Factor  Title:  INVALID  CBECKS  HADE 


Factor  Description/Definition;  Making  a  wrong  test  at  a  test  point 

given  the  status  of  the  circuit  or  the 
condition  of  the  hardware. 


Example  of  Factor  Application:  The  troubleshooter  measures  current 

where  he  should  have  been  checking  for 
voltage . 


TRQDBLESBQOTING  PROFICIENCY  EVALPATION  PROGRAM  (TPEP) 


Level  of  Importance 
1-2-3-4-5-6-7-6-9-10 
Kost  -  -  Medium  -  -  Least 
(circle  1  per  card) 


Factor  Title:  ILLOGICAL  APPROACB 


Factor  Description/Definition;  Testing  begins  at  a  point  not  indicated 

by  the  symptoms.  The  troubleshooter  may 
still  diagnose  the  fault,  but  does  not 
efficiently  utilize  the  symptom  data. 


Example  of  Factor  Application:  If  the  troubleshooter  begins  testing  on 

UNIT  7,  even  though  iJJL  the  fault 
symptoms  and  indicators  (including  unit 
7)  point  to  UNIT  1  as  the  probable  fault 
source  this  is  an  "Illogical 
Approach."  This  applies  even  if  the 
trouble  could  be  found  following  this 
"illogical  approach." 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
^  Most  -  -  Medium  -  -  Least 

(circle  1  per  card) 

Factor  Title:  OUT  OP  BOUNDS 


Factor  Description/Definition:  Number  of  test  points  selected  that  were 

not  relevant  to  diagnosing  the  fault. 


Example  of  Factor  Application:  The  person  troubleshooting  a  system 

makes  tests  that  are  not  reasonably  in 
the  area  of  where  the  trouble  is 
actually  located.  The  troubleshooter 
makes  tests  at  points  that  have  nothing 
to  do  with  the  problem. 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
Most  -  -  Medium  -  -  Least 
(circle  1  per  card) 

Factor  Title:  NUMBER  OP  TEST  POINTS  CHECKED 


Factor  Description/Definition:  Number  of  test  points,  card  pins, 

terminal  board  pins  examined  to  isolate 
the  fault. 


Example  of  Factor  Application:  What  is  the  total  number  of  tests  a 

troubleshooter  makes  in  determining  the 
trouble?  The  troubleshooter  finds  the 
solution  by  checking  the  minimum  number 
of  points. 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
^  Most  -  -  Medium  -  -  Least 

(circle  1  per  card) 

Factor  Title:  COST 


Factor  Description/Definition:  Number  of  components,  cards  or  units 

incorrectly  identified  and  replaced  as 
being  the  fault  source. 


Example  of  Factor  Application:  A  troubleshooter  replaces  3  cards  to 

repair  the  equipment  when  only  1  card 
was  bad. 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
Most  -  -  Medium  -  -  Least 
(circle  1  per  card) 

Factor  Title:  PROOF  POINTS  IDENTIFIED 


Factor  Description/Definition:  Number  of  possible  input  and  output 

points  of  faulty  circuit  that  were 
tested. 


Example  of  Factor  Application:  A  solution  is  "proven"  when  the  minimum 

number  of  proof  points  have  been  tested 
which  conclusively  isolate  the  faulty 
component.  The  troubleshooter  must 
"prove"  what  he  was  replacing  was  the 
fault? 


TRODBLESBQQTING  PROFICIENCY  EVALOATION  PROGRAM  fTPEP^ 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
Most  -  -  Medium  -  -  Least 
(circle  1  per  card) 


Factor  Title;  SOLUTION 


Factor  Desci iption/Def inition :  Troubleshooter  identifies  the  fault 

source/component  -  tech  finds  the 
trouble. 


Example  of  Factor  Application;  The  person  troubleshooting  a  system 

does,  in  fact,  find  the  cause  of  the 
trouble  (e.g.,  circuit  card,  component, 
etc. ) . 


I 

TROUBLESHOOTING  PROFICIENCY  ■EVALUATION-fRQGRAM_(TP£Pl 


Level  of  Importance 
1-2-3-4-5-6-7-8-9-10 
Most  -  -  Medium  -  -  Least 

(circle  1  per  card) 

Factor  Title;  TIKE 


Factor  Description/Definition:  Time  (in  minutes)  it  takes  to  isolate 

and  identify  the  fault. 


Example  of  Factor  Application:  The  amount  of  time  it  takes  for  the 

troubleshooter  to  discover  the  cause 
(solution)  of  the  fault. 


I 
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DESCRIPTIVE  STATISTICS 


HEADER  DATA  FOR:  C;TPEP3  LABEL:  TPEP  SURVEY  HASTER  FILE 
NUMBER  OF  CASES:  750  NUMBER  OF  VARIABLES:  27 

TAE  FACTOR  RANKING 


NO. 

NAME 

N 

MEAN 

STD.  DEV. 

HINIMUn 

1 

»sn 

750 

NA 

NA 

NA 

2 

750 

NA 

NA 

NA 

3 

•8* 

750 

31.6453 

10.9215 

.0000 

4 

rat 

750 

1.1733 

.3788 

1.0000 

c 

pa* 

750 

6.5733 

.8001 

6.0000 

6 

sex 

750 

1.0387 

.1929 

1.0000 

7 

•fit 

750 

1.3147 

.7539 

.0000 

8 

dut 

750 

1.8813 

1.1324 

1.0000 

9 

rac 

750 

1.5053 

.6312 

1.0000 

10 

Loe 

750 

3.1347 

1.0135 

1.0000 

11 

aoe 

750 

13.1000 

9.6243 

.0000 

12 

job 

750 

3.5427 

3.7657 

.0000 

13 

tis 

750 

152.4707 

74.6404 

.0000 

14 

tad 

750 

41.4880 

31.5327 

.0000 

15 

nec 

750 

1.6507 

.9504 

.0000 

16 

afq 

750 

74.5973 

24.9347 

.0000 

17 

aef 

750 

104.3373 

121.1124 

.0000 

18 

fOI 

750 

2.3267 

2.3256 

.0000 

19 

f02 

750 

5.2600 

2.8096 

.0000 

20 

f03 

750 

5.5053 

2.7332 

.0000 

21 

t04 

750 

4.8960 

2.3482 

.0000 

22 

f05 

750 

7.1333 

2.4118 

.0000 

23 

f06 

750 

6.3880 

2.4593 

.0000 

24 

■f07 

750 

6.9493 

2.2292 

.0000 

25 

f08 

750 

6.0667 

2.7095 

.0000 

26 

f09 

750 

3.6413 

2.5374 

.0000 

27 

tio 

750 

6.5907 

2.3673 

.0000 
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i-10-88 


MAXIMUM 

NA 

NA 

57.0000 

2.0000 

9.0000 

2.0000 

2.0000 

4.0000 

5.0000 

7.0000 

42.0000 

22.0000 

355.0000 

218.0000 

3.0000 

99.0000 

285.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 


APPENDIX  D 

FACTOR  WEIGHING  QUESTIONNAIRE 
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NAVY  PERSONNEL  RESEARCH  AND  DEVELOPMENT  CENTER 
TROUBLESHOOTING  ASSESSMENT  AND  ENHANCEMENT 


INTRODUCTION 

The  Chief  of  Naval  Operations  (via  the  Vice  Chief  of  Naval  Operations  for 
Manpower,  Personnel  and  Training)  has  stated  a  requirement  of  developing  a 
reliat’.e  and  objective  method  for  assessing  troubleshooting  skills  of  Navy 
technicians.  The  Navy  Personnel  Research  and  Development  Center  (NPRDC),  San 
Diego,  California  has  been  tasked  to  collect  information  and  carry  out  this 
requirement.  The  name  of  the  research  and  development  project  that  is 
accomplishing  this  tasking  is  the  “Troubleshooting  Assessment  and  Enhancement" 

•:  tae  ) . 

Previously  a  questionnaire  was  administered  to  over  700  Navy  technicians  in  an 
effort  to  get  an  idea  of  the  things  that  should  be  considered  in  evaluating 
whether  a  technician  is  a  "good"  troubleshooter  or  not.  Also,  these 
technicians  were  asked  to  rank  order  the  "factors"  from  most  important  to  least 
important . 

Now  it  is  necessary  to  evaluate  these  "factors"  as  they  have  been  "ranked"  and 
to  determine  the  "weight"  that  should  be  applied  to  each  factor.  You  have  been 
selected  to  participate  in  this  effort  because  you  are  considered  a  Subject 
Matter  Expert  (SHE)  in  troubleshooting  Navy  systems  and  equipment.  The 
information  that  you  provide  will  be  valuable  and  will  be  used  as  the  basis  for 
developing  a  method  of  assessing  and  enhancing  the  troubleshooting  skills  of 
Navy  personnel. 

Please  keep  in  mind,  that  this  is  not  a  test,  nor  will  the  information  you 
provide  be  released  to  persons  other  than  yourself  or  those  directly  concerned 
with  the  project.  You  c  your  career  will  not  be  jeopardized  in  any  way. 

GENERAL  INSTRUCTIONS 


1.  Please  read,  date,  provide  SSN  and  sign  "Privacy  Act  Statement"  provided 

on  response  sheet. 

2.  Please  read  the  following  instructions  for  each  section  of  the  response 
sheet  before  responding  to  any  items. 

2.  If  you  wish  to  change  a  response,  be  sure  to  completaly  erase  (or  clearly 
mark  out  if  using  pen)  any  previous  response. 

rROUBLESHCX>TING  ASSESSMENT  i  ENHANCEMENT  rTAE)  INFORMATION 

The  list  of  "ranked"  factors  supplied  on  the  response  sheet  concern  the 
factors  that  previous  subject  matter  experts  have  determined  to  be  related  to 
the  assessment  of  troubleshooting  skills.  A  description  and  explanation  of  the 
ranked  factors  is  provided  below.  Please  read  all  the  following  information 
and  each  of  the  "factor"  descriptions  before  proceeding  with  the  "weighing"  of 
factors  on  the  response  sheet. 
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LIMITATION  OF  THE  TROUBI.KSHOOTTWG  rACTORS 

It  is  understood  that  there  are  a  number  of  factors  that  are  involved  in 
assessing  a  troubleshooters  capability  and  that  these  factors  vary  in 
importance  (i.e.,  they  have  different  weights).  In  this  "weighing"  the  factors 
have  been  restricted,  intentionally,  for  varying  reasons  (e.g.,  safety  for 
personnel  and  hardware  are  not  included  for  it  is  understood  that  if  you 
violate  safety  you  are  a  "bad"  troubleshooter).  Also,  use  of  test  equipment, 
soldering,  and  other  "support"  skills  are  not  included.  These  are  considered 
Bupjxjrt  ar.d  can  be  measured  via  other  methods.  "Troubleshooting,"  however,  is 
considered  to  be  a  mental  effort  and  factors  are  required  that  can  be  checked 
to  see  if  the  mental  process  of  one  troubleshooter  is  more  efficient  than 
another.  Figure  1  shows  the  factors  that  you  are  being  asked  to  weigh. 

CONDITIONS  OF  TROUBLESHOOTING  gPISODE 

In  selecting,  ranking  .ind  "weighing"  of  factors  that  one  would  use  in 
determining  (assessing)  if  one  troubleshooter  was  more  efficient  than  another 
it  is  necessary  that  the  environment  (e.g.,  war  or  peace,  at  sea  or  tied  up, 
etc.)  of  the  troubleshooting  effort  be  identified.  If  the  environment  changes 
this  may  change  the  results  . 

For  purposes  of  this  "weighing"  effort  assume  the  following  environment  and/or 
conditions : 

a.  NON-COMBAT  (i.e.  peace  time) 

b.  NORMAL  DAY  IN  HOME-PORT 

C.  DURING  A  DAILY  CHECK  OF  THE  EQUIPMENT  A  TROUBLE  IS  INDICATED 


SOLUTI ON 


PROOF  POI NTS 
COST 


**  OF  TEST  POI  NTS' 

W  INUALID  TEST 


TI  ME 

ILLOGICAL  APPROACHES 


«  OF  CHECKS 
«  OUT  OF  BOUNDS 


FIGURE  1.  TROUBLESHOOTING  FACTORS  TO  BE  WEIGHTED 
FACTOR  WEIGHING  INSTRUCTIONS 

After  reading  the  information  above  and  reviewing  all  the  factors  descriptions 
please  perform  the  following  steps: 

1.  Weigh  each  factor  (in  terms  of  100%)  according  to  how  important  the  factor 
described  is  in  assessing  troubleshooting  ability. 

2.  Once  all  the  factors  are  weighed  please  add  the  percentages.  Ensure  the 

result  equals  100%.  If  the  total  is  not  100%  please  modify  your 
weightings  until  the  total  does  sum  to  100%.  Feel  free  to  change  your 
weightings  as  you  wish. 
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TAB  FACTOR  OEFINITION/EZAMPLE  INFORMATION 


FACTOR 


SOLUTION 


PRO.~r 

POINTS 


TIKE 


COST 


ILLOGICAL 

APPROACH 


NUMBER  OF 
TEST  POINTS 
CHECKED 


NUMBER  OF 
CHECKS  MADE 


IN-v'ALID 
CHECKS  KAJ3E 


OUT  OF 
BOUNDS 


DESCRIPTION 


EXAMPLE 


Troubleshooter  identifies  the 
fault  source  nd/or  comjxsnent. 
Tech  finds  the  trouble. 


Number  of  possible  input  and 
output  points  of  faulty  circuit 
that  were  tested. 


Time  (in  minutes)  it  takes  to 
isolate  and  identify  the  fault. 


Nurrloer  of  components,  cards  or 
units  incorrectly  identified 
and  replaced  as  being  the  fault 
source . 

Testing  begins  at  a  point  not 
indicated  by  the  symptoms.  The 
troubleshooter  may  still 
diagnose  the  fault,  but  does 
not  efficiently  utilize  the 
symptom  data. 


Number  of  test  points,  card 
pins,  terminal  board  pins 
examined  to  isolate  the  fault. 


Number  of  checks  performed  to 
isolate  the  fault.  These 
include  continuity,  logic, 
frequency,  current,  voltage  and 
waveforms . 


Making  a  wrong  test  at  a  test 
point  given  the  status  of  the 
circuit  or  the  condition  of  the 
hardware . 

Number  of  test  points  selected 
that  were  not  relevant  to 
diagnosing  the  fault. 


The  person  troubleshooting  a 
system  does,  in  fact,  find  the 
cause  of  the  trouble  (e.g., 
circuit  card,  component,  etc.). 

A  solution  is  "proven"  when  the 
minimum  number  of  proof  points 
have  been  tested  which 
conclusively  isolate  the  faulty 
component.  The  troubleshooter 
must  "prove"  what  he  was 
replacing  was  the  fault? 

The  aunount  of  time  it  takes  for 
the  troubleshooter  to  discover 
the  cause  (solution)  of  the 
fault . 

A  troubleshooter  replaces  3 
cards  to  repair  the  equipment 
when  only  1  card  was  bad. 


If  the  troubleshooter  begins 
testing  on  UNIT  7,  even  though 
all  the  fault  symptoms  and 
indicators  (including  unit  7) 
point  to  UNIT  1  as  the  probable 
fault  source  this  is  an 
"Illogical  Approach."  This 
applies  even  if  the  trouble 
could  be  found  following  this 
"illogical  approach.” 

What  is  the  total  number  of 
tests  a  troubleshooter  makes  in 
determining  the  trouble?  The 
troubleshooter  finds  the  solu¬ 
tion  by  checking  the  minimum 
number  of  points. 

The  troubleshooter  does  not 
make  a  number  of  different 
tests  (logic,  voltage,  waveform 
etc.)  at  the  same  test  point 
using  different  test  equipment 
when  one  test  provides  the 
information  needed. 

The  troubleshooter  measures 
current  where  he  should  have 
been  checking  for  voltage. 


The  person  troubleshooting  a 
system  makes  tests  that  are  not 
reasonably  in  the  area  of  where 
the  trouble  is  actually 
located.  The  troubleshooter 
makes  tests  at  points  that  have 
nothing  to  do  with  the  problem. 
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FACTOR  HEZOBIMO  RESPONSE  SHEET 


PRIVACY  ACT  STATEMENT;  You  have  been  selected  to  participate  in  the  TAE 
project.  This  project  provides  research  data  on  the  different  levels  of 
troubleshooting  expertise  associated  with  Navy  systems  and  equipment.  The 
information  provided  by  you  will  be  used  by  the  Navy  Personnel  Research  and 
Development  Center,  San  Diego,  for  research  purposes  only.  It  will  not  become 
a  part  of  your  official  record,  nor  will  it  be  used  to  make  decisions  about  you 
which  will  affect  your  career  in  any  way.  Your  name,  SSN  are  necessary  only  to 
aid  in  processing  the  research  data. 

Date:  _  SSN:  _ 

Signature:  _ 

BACKGROUND  INFORMATION:  The  following  items  concern  your  general  background. 

Please  provide  information  as  indicated  (please  print). 

1. 


FACTOR  VfEIGHING;  Please  ensure  that  the  sum  of  the  factors  totals  100%  —  if 
not  please  re-weigh  the  factors  until  you  achieve  the  100%  sum. 


RANKING 

FACTOR 

WEIGHT 

(in  percentage) 

RANKED 

#1  . 

SOLUTION 

% 

RANKED 

#2. 

PROOF  POINTS 

% 

RANKED 

#3  . 

TIME 

% 

RANKED 

/4. 

COST 

% 

RANKED 

F5. 

ILLOGICAL  APPROACH 

% 

RANKED 

/6. 

NUMBER  OF  TEST  POINTS 

% 

RANKED 

#7. 

NUMBER  OF  CHECKS 

% 

RANKED 

#8. 

INVALID  CHECKS 

% 

RANKED 

OUT  OF  BOUNDS 

% 

RANKED 

/lO. 

OTHER 

% 

SUM  OF 

FACTORS 

(write  in) 

1  0  0  % 

UPON  COMPLETION  PLEASE  PLACE  QUESTIONNAIRE  PACKET  IN  AN  ENVELOPE  AND  NAIL  TO: 

TAE  PROJECT,  CODE  142 

NAVPERSRANDCEN 

SAN  DIEGO,  CA  92152-6800 


TAE  FACTOR  WEIGHING  DATA 


HEADER  DATA  FOR:  C:FACWEI62  LABEL:  TAE  FACTOR  WEIGHING 
NUMBER  OF  CASES;  45  NUMBER  OF  VARIABLES:  17 


:s>. 

JOB 

RATE/ING 

LOCATION 

FAISOL 

FA2PRP 

FA3TIM 

4.00 

9.00 

1.x 

50.x 

.X 

20.x 

Ztttttttm 

1.00 

7.00 

1.x 

35.x 

25.x 

4.x 

J.lrmnrtrt'r  xnit 

1.00 

6.00 

1.x 

30.x 

20.x 

8.x 

4tl^Xrirtiri(incieic 

1.00 

7.00 

1.x 

40.x 

20.x 

20.x 

%*1eirxicir*^x 

1.00 

6.00 

1.x 

20.x 

15.x 

5.x 

(yinriririrtii  *  Irk 

1.00 

5.00 

1.x 

70.x 

10.x 

5.x 

^Irkkktktt-kk 

1.00 

6.00 

1.x 

40.x 

10.x 

15.x 

^kkkktkkkkk 

2.00 

6.00 

1.x 

40.x 

20.x 

5.x 

<)kkkkkk*1tkk 

1.00 

6.00 

1.x 

25.x 

15.x 

10.x 

10****«****« 

1.00 

6.00 

1.x 

30.x 

15.x 

10.x 

^^1^kirki^kir^^lrk 

2.00 

7.00 

1.x 

70.x 

5.x 

10.x 

12***«****** 

1.00 

6.00 

1.x 

70.x 

.X 

2.x 

13**o******» 

1.00 

6.00 

1.x 

70.x 

.X 

15.x 

4irtr-<nrtnHrtfTHr 

1.00 

6.00 

1.x 

X.X 

10.x 

6.x 

^Skkk**k*irtk 

1 .00 

6.00 

1.x 

70.x 

.X 

5.x 

16*  kkkkkirkkk 

1.00 

6.00 

1.x 

45.x 

5.x 

18.x 

17*A<I*)I****> 

1.00 

6.00 

1.x 

50.x 

5.x 

10.x 

IQirtinttHrtrtrtr* 

1.00 

6.00 

1.x 

45.x 

5.x 

15.x 

19**jt****-*"** 

1.00 

6.00 

1.x 

40.x 

.X 

30.  X 

20***  ****"*•*> 

1.00 

6.00 

1.x 

50.x 

.X 

.X 

ZHtttHtAAtt 

1.00 

6.00 

1.x 

60.x 

5.x 

10.x 

22********** 

1.00 

6.00 

1.x 

40.x 

15.x 

5.x 

23******»*** 

3.00 

6.00 

1.x 

50.x 

20.x 

.X 

24*+**irtHhhrt 

1.00 

6.00 

1.x 

70.x 

5.x 

5.x 

25********^ 

1.00 

6.00 

1.x 

50.x 

10.x 

15.x 

26«*«*«***a« 

3.00 

7.00 

1.x 

60.x 

2.50 

15.x 

27********iMt 

3.00 

7.00 

1,X 

.X 

15.x 

30.x 

28***«««***« 

1.00 

6.00 

1.x 

25.x 

10.x 

15.x 

29********** 

1.00 

6.00 

1.x 

20.x 

15.x 

12.x 

1.00 

6.00 

1.x 

35.x 

5.x 

12.x 

31********** 

1.00 

6.00 

1.x 

20.x 

10.x 

10.x 

32********** 

1.00 

7.00 

1.x 

20.x 

10.x 

in  X 

33ir********* 

2.00 

6.00 

1.x 

50.x 

5.x 

8.x 

34 *★****»«*« 

1.00 

6.00 

1.x 

90.x 

.X 

5.x 

3^*****>»**> 

2.00 

6.00 

1.x 

30.x 

20.x 

15.x 

36***»*>***^ 

2.00 

6.00 

1.x 

30.x 

10.x 

6.x 

37t*«t*«*»** 

1.00 

6.00 

1.x 

20.x 

15.x 

10.x 

35<**<«<y»*ii 

4.00 

7.00 

1.x 

30.x 

5.x 

15.x 

39********** 

5.00 

9.00 

1.x 

30.x 

20.x 

15.x 

40********** 

1.00 

6.x 

1.x 

50  X 

1.x 

30.x 

41********** 

1.00 

6.x 

1.x 

50.x 

15.x 

20.x 

42***^  ****** 

1.00 

6.x 

1.x 

30.x 

15.x 

20.x 

43********** 

4.00 

5.x 

1.x 

20.x 

25.x 

10.x 

44********** 

2.00 

6.x 

1.x 

50.x 

.X 

20.x 

45********** 

1.00 

6.x 

1.x 

25.x 

10.x 

5.x 
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FA4C0S 

fA5ILA 

FA6NUT 

FA7NUC 

FA8INC 

FA900B 

FA1XTH1 

1 

30,00 

.X 

.X 

.X 

.X 

.X 

.X 

2 

10.00 

10.x 

2.x 

2.x 

10.x 

2.00 

.X 

3 

14.00 

8.x 

4.x 

S.X 

8.x 

3.x 

.X 

4 

10.00 

2.x 

2.x 

2.x 

2.x 

2.x 

.X 

5 

10.00 

15.x 

5.x 

10.x 

10.x 

10.x 

.X 

6 

15.00 

.X 

.X 

.X 

.X 

.X 

.X 

7 

10  00 

5.x 

5.x 

S.X 

S.X 

S.X 

.X 

6 

10.00 

.X 

S.X 

S.X 

S.X 

S.X 

.X 

9 

5.00 

10.x 

10.x 

S.X 

S.X 

10.x 

S.X 

1C 

15. OC 

10.x 

.X 

.X 

10.x 

10.x 

.X 

■  1 

5.00 

.X 

2.50 

2.50 

2.50 

2.50 

.00 

•2 

10. OC 

2.x 

3.x 

S.X 

3.x 

3.x 

.00 

13 

15.00 

.X 

.X 

.X 

.X 

.X 

.00 

14 

.00 

.X 

1.x 

1.x 

.X 

2.x 

.00 

15 

20,00 

.X 

.X 

.X 

.X 

.X 

.00 

16 

20.00 

5.x 

1.x 

3.x 

1.x 

2.x 

.00 

17 

10.00 

S.X 

S.X 

S.X 

S.X 

S.X 

.00 

18 

10.00 

5.x 

S.X 

S.X 

S.X 

S.X 

.00 

19 

30.00 

.  X 

.X 

.X 

.X 

.X 

.00 

20 

30.00 

10.  O 

.X 

.X 

10. X 

.X 

.00 

21 

5.00 

3.x 

S.X 

10.x 

1.x 

1.x 

.00 

22 

10.00 

15  X 

2.50 

2.50 

S.X 

S.X 

.00 

23 

5.00 

5  X 

S.X 

S.X 

S.X 

S.X 

.00 

24 

20.00 

M 

-X 

.X 

.X 

.X 

.00 

25 

2.00 

5  :o 

S.X 

S.X 

S.X 

3.x 

.X 

26 

15.00 

X 

.X 

.00 

S.X 

2.50 

.X 

27 

20.00 

10.x 

S.X 

S.X 

S.X 

10.x 

.00 

28 

20.00 

10  X 

S.X 

8.x 

6.x 

1.x 

.00 

29 

15.00 

10  X 

S.X 

S.X 

10.x 

10.x 

.00 

30 

10.00 

17  X 

3.x 

3.x 

4.x 

11.x 

.00 

31 

2.00 

15  X 

10.x 

10.x 

13.x 

10.x 

.00 

32 

20.00 

10  X 

10.x 

S.X 

10.x 

S.X 

.00 

33 

8,00 

10  X 

3.x 

3.x 

S.X 

8.x 

.00 

34 

5.00 

X 

.X 

.X 

.X 

.X 

.00 

35 

10.00 

3  X 

1.x 

S.X 

7.x 

2.x 

7.00 

36 

5.00 

20  X 

7.x 

2.x 

13.x 

7.x 

.00 

37 

5.00 

25  X 

10.x 

S.X 

S.X 

S.X 

.00 

38 

30.00 

5  X 

2.50 

2.50 

5  00 

S.X 

.00 

39 

15.00 

8.x 

6.x 

2.x 

2.x 

2.x 

.00 

40 

10  00 

1.x 

1.x 

S.X 

1.x 

i.X 

.00 

41 

15.00 

.X 

.X 

.X 

.X 

.X 

.00 

42 

15.x 

10.x 

.X 

.X 

S.X 

S.X 

.X 

43 

5.x 

x.x 

.X 

.X 

.X 

S.X 

.X 

4i. 

30  X 

.X 

.X 

.00 

.00 

.X 

.X 

4'. 

15.x 

10.x 

10.x 

.00 

15.x 

10.x 

.X 
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DESCRIPTIVf:  STATISTICS 


HFADEIR  DATA  tOR:  C:FAC\'.T:IC2  LADCL:  TAE  FACTOR  VTTIGHING 
riUMBLR  OF  CASF^:  45  fJUfBER  OF  VARIABLES:  14 

TAE  FACTOR  KEiailUG  RESULTS 


NO. 

NA‘1F 

N 

MFAN 

1 

SSN 

45 

2 

J(  'B 

45 

1.5556 

3 

RATE/INC 

45 

6.2444 

4 

LOCATION 

45 

1,0000 

5 

FA IDOL 

45 

42.7778 

6 

FA2PRP 

45 

9.8556 

7 

FA3TIM 

45 

11.8000 

8 

FA4COS 

45 

13.1333 

9 

FA5ILA 

45 

6.8667 

10 

FA6NtIT 

45 

3.2111 

11 

FA7^WC 

45 

3.0778 

12 

FAHir^ 

45 

4.6778 

13 

FA900B 

45 

4.0000 

14 

FAIOOTHI 

45 

.2667 

STD.  DEV. 

MINI-IIJM 

MAXIMUM 

1.0347 

1,0000 

5.0000 

.7433 

5.0000 

9,0000 

.0000 

1.0000 

1.0000 

19.2636 

.0000 

90.0000 

7.4411 

.0000 

25.0000 

7.3719 

.0000 

30.0000 

8.0216 

.0000 

30.0000 

7.0666 

.0000 

30.0000 

3.2113 

.0000 

10.0000 

2.9212 

.0000 

10.0000 

4.0887 

.0000 

15.0000 

3.5532 

.0000 

11.0000 

1.2685 

.0000 

7.0000 

D-7 
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